Comparative judgment is gaining popularity as an assessment tool, including for high-stakes testing purposes, despite relatively little research on the use of the technique. Advocates claim two main rationales for its use: that comparative judgment is valid because humans are better at comparative than absolute judgment, and because it distils the…

This article discusses how comparability relates to current mainstream conceptions of validity, in the context of educational assessment. Relevant literature was used to consider the relationship between these concepts. The article concludes that, depending on the exact claims being made about the appropriate interpretations and uses of the…

There is increased emphasis on using experimental and quasi-experimental methods to evaluate educational programs; however, educational evaluators and school leaders are often faced with challenges when implementing such designs in educational settings. Use of a historical cohort control group design provides a viable option for conducting…

The overclaiming technique is a novel assessment procedure that uses signal detection analysis to generate indices of knowledge accuracy (OC-accuracy) and self-enhancement (OC-bias). The technique has previously shown robustness over varied knowledge domains as well as low reactivity across administration contexts. Here we compared the OC-accuracy…

As the US continues to diversify, methods for accurately assessing human behavior must evolve. This paper offers multicultural research considerations at several stages of the research process for psychological research and consultation. Implications regarding the comparative research framework are discussed and suggestions are offered on how to…

In recent guidelines for fair educational testing it is advised to check the validity of individual test scores through the use of person-fit statistics. For practitioners it is unclear on the basis of the existing literature which statistic to use. An overview of relatively simple existing nonparametric approaches to identify atypical response…

Although International Association for the Evaluation of Educational Achievement-pioneered international large-scale assessment (ILSA) of education is now a well-established science, non-practitioners and many users often substantially misunderstand how large-scale assessments are conducted, what questions and challenges they are designed to…

Comparability between different educational qualifications is an important issue within policy discourse in the UK. In this context, the comparability of qualification demands has been explored through the use of expert human judgement. The involvement of human judgement in estimating assessment demands has consequences for methodology. This…

This study provided preliminary evidence for the reliability and validity of "Teaching Strategies GOLD", a recently developed observational system for assessing young children's development and learning. The measurement properties of "Teaching Strategies GOLD" were compared with those of an older instrument, "The Creative…

This study examines student growth on mathematics and reading assessments across academic years (Spring 2006 through Spring 2009) using three different growth models: hierarchical linear model (HLM), value-added model (VAM), and student growth percentile model (SGP). Comparisons across these three growth models were conducted to investigate the…

This study used a Chinese-language version of the Index of Science Reading Awareness to explore the science reading metacognition and comprehension of Taiwanese students. Structural equation modelling results confirmed the underlying model comprised three clusters of metacognitive knowledge: beliefs and confidence in science reading, knowledge of…

In this paper, we examine the validity and precision of two nonexperimental study designs (NXDs) that can be used in educational evaluation: the comparative interrupted time series (CITS) design and the difference-in-difference (DD) design. In a CITS design, program impacts are evaluated by looking at whether the treatment group deviates from its…

The 2013-2014 North Carolina Pre-Kindergarten (NC Pre-K) Evaluation study was designed to examine the longitudinal outcomes through kindergarten for children who attended the Pre-K program, along with comparisons to previous cohorts of program attendees. A sample of 561 children was included in the study, with data gathered at the beginning and…

Introduction: For decades, prior to the inception of the Common Core State Standards (CCSS), the National Assessment of Educational Progress (NAEP) was the only vehicle through which states could assess the progress of their students using a common metric. Now, 45 states, 4 U.S. territories, and the District of Columbia have adopted the CCSS to…

The Yes/No Angoff and Bookmark method for setting standards on educational assessment are currently two of the most popular standard-setting methods. However, there is no research into the comparability of these two methods in the context of language assessment. This study compared results from the Yes/No Angoff and Bookmark methods as applied to…