Quantitative measures of systematic differences in OSCE scoring across examiners (often termed examiner stringency) can threaten the validity of examination outcomes. Such effects are usually conceptualised and operationalised based solely on checklist/domain scores in a station, and global grades are not often used in this type of analysis. In…

In this paper, we explore measurements of design creativity through metrics related to the processes used in designing and relate them to the metrics used in psychology for idea creativity, ie, novelty and fluency. Our goal was to test the reliability of psychometric measures of creativity to assess creativity in team design. We studied 19 teams…

Given the lack of cultural consideration of studies on digital natives, this study reports on a large-scale validation of the Digital Native Assessment Scale (DNAS) among university students from three regions of Greater China: Chinese mainland, Macau, and Taiwan, to examine measurement invariance and latent mean differences in the four constructs…

Education researchers often encounter scenarios where an abrupt policy change occurs within or across jurisdictions or populations that affect key student outcomes. Difference-in-differences is a research design analysts can use to estimate causal effects of these "natural experiments." This article introduces education researchers to…

As evidence from evaluation and experimental studies continue to influence decision and policymaking, applied researchers and practitioners require tools to derive valid and credible inferences. Over the past several decades, research in causal inference has progressed with the development and application of propensity scores. Since their…

A mixed methods randomized control trial (RCT) was employed to investigate the social validity ratings (i.e., acceptability, appropriateness, and feasibility) of middle-school students using an innovative program of virtual social scenarios called VOISS (Virtual reality (VR) Opportunities to Integrate Social Skills) compared to an evidence-based,…

In this study, a valid and reliable scale (TSSSSS) was aimed to develop to measure secondary school students' perceptions of teacher support. The research was carried out on a total of 773 students studying in 6 different secondary schools in Ankara's Yenimahalle district in the Spring Term of the 2021-2022 Academic Year. The students who…

Universal screening to predict students' risk for reading problems is a foundational component of the Multi-Tiered Systems of Support framework and is required by law in many US states. School or district administrators are tasked with selecting screening assessments that are both technically adequate and feasible given the resources of their…

Despite agreement about the central importance of validity for educational and psychological testing, consensus regarding the definition of validity remains elusive. Differences in the definition of validity are examined and reveals that a potential cause of disagreement stems from differences in word use and meanings given to key terms commonly…

This module is designed for educators, educational researchers, and psychometricians who would like to develop an understanding of the basic concepts of validity theory, test validation, and documenting a "validity argument." It also describes how an in-depth understanding of the purposes and uses of educational tests sets the foundation…

In this article, we argue that automated scoring engines should be transparent and construct relevant--that is, as much as is currently feasible. Many current automated scoring engines cannot achieve high degrees of scoring accuracy without allowing in some features that may not be easily explained and understood and may not be obviously and…

A critical issue in using fine-grained gameplay data to measure learning processes is the development of indicators and the algorithms used to derive such indicators. Successful development--that is, developing traceable, interpretable, and sensitive-to-learning indicators--requires understanding the underlying theory, how the theory is…

The purpose of this paper is to present a conversation about how impact has been seen and felt by youth and adult co-researchers in a long-term, school-based youth participatory action research (YPAR) project, "Courageous Conversations." To stay true to the epistemological commitments of YPAR, and in an effort to speak back to…

Active learning is a popular approach to teaching and learning that has gained traction through research on STEM educational improvement. There have been numerous university- and national/international-level efforts focused on transitioning courses from the lecture method to active learning. However, despite these large-scale changes, the active…

In this study, we aimed to find problematic and supportive issues in Afghan undergraduate students' proofs of the irrationality of [square root]3 and [square root]5/8 while using the indirect proof method. Collecting and analyzing produced proofs of 30 sophomore and 48 senior undergraduate students on the irrationality of [square root]3 and…