This article presents the performance of item response theory (IRT) models when double ratings are used as item scores over single ratings when rater effects are present. Study 1 examined the influence of the number of ratings on the accuracy of proficiency estimation in the generalized partial credit model (GPCM). Study 2 compared the accuracy of…

As the development and application of large language models (LLMs) in physics education progress, the well-known AI-based chatbot ChatGPT4 has presented numerous opportunities for educational assessment. Investigating the potential of AI tools in practical educational assessment carries profound significance. This study explored the comparative…

This National Academy of Education (NAEd) volume provides guidance to key stakeholders on how to accurately report and interpret comparability assertions concerning large-scale educational assessments as well as how to ensure greater comparability by paying close attention to key aspects of assessment design, content, and procedures. The goal of…

This article discusses how comparability relates to current mainstream conceptions of validity, in the context of educational assessment. Relevant literature was used to consider the relationship between these concepts. The article concludes that, depending on the exact claims being made about the appropriate interpretations and uses of the…

This dissertation investigates the application of network models in educational assessment across diverse learning settings, encompassing both clinical and classroom environments. As educational paradigms shift and digital tools become more embedded in learning processes, traditional assessment methods often fall short in effectively capturing the…

Distractor analyses are routinely conducted in educational assessments with multiple-choice items. In this research report, we focus on three item response models for distractors: (a) the traditional nominal response (NR) model, (b) a combination of a two-parameter logistic model for item scores and a NR model for selections of incorrect…

As the theme of the 2022 annual meeting of the American Education Research Association, cultivating equitable education systems has gained renewed attention amid an increasingly diverse society. However, systemic inequalities persist for traditionally underserved student populations. As a way to better address diverse students' needs, it is of…

The purpose of this causal comparative study was to test the theory of assessment that relates benchmark assessments to the Georgia middle grades science Criterion Referenced Competency Test (CRCT) percentages, controlling for schools who do not administer benchmark assessments versus schools who do administer benchmark assessments for all middle…

Purpose: This paper aims to report on a census of high-level sustainability initiatives at all accredited post-secondary institutions in Canada by documenting the institutions that have undertaken sustainability assessments, have signed one or more sustainability declarations, have sustainability offices or officers or have sustainability…

In recent guidelines for fair educational testing it is advised to check the validity of individual test scores through the use of person-fit statistics. For practitioners it is unclear on the basis of the existing literature which statistic to use. An overview of relatively simple existing nonparametric approaches to identify atypical response…

Hong Kong (HK) and the People's Republic of China (PRC), while sharing historic cultural roots, have different policies for and practices of educational assessment. Student conceptions of assessment function to guide individual behaviour in response to the functions, purposes, and consequences of assessments. A new self-report questionnaire was…

This is the Mackinac Center's fourth school report card and covers elementary and middle schools. A similar report card was published in 2013, and this edition includes two years' worth of new data. A unique characteristic of this report card is that takes into consideration the "context" of a school when assessing its performance.…

Trying to understand PISA is analogous to the parable of the blind men and the elephant. There are many facets of the PISA program, and thus many ways to both applaud and critique this ambitious international program of assessment that has gained enormous importance in the crafting of contemporary educational policy. One of the facets discussed in…

Results from international large-scale assessments (ILSAs) garner considerable attention in the media, academia, and among policy makers. Although there is widespread recognition that ILSAs can provide useful information, there is debate about what types of comparisons are the most meaningful and what could be done to assure more sound…

Many US education reform efforts focus on student performance in large, urban school districts. The National Assessment of Educational Progress's Trial Urban District Assessment (TUDA) program provides data on student achievement in these districts, but differences in student characteristics complicate comparisons of district performance. I use…