Topic models are mathematical and statistical models used to analyze textual data. The objective of topic models is to gain information about the latent semantic space of a set of related textual data. The semantic space of a set of textual data contains the relationship between documents and words and how they are used. Topic models are becoming…

As the development and application of large language models (LLMs) in physics education progress, the well-known AI-based chatbot ChatGPT4 has presented numerous opportunities for educational assessment. Investigating the potential of AI tools in practical educational assessment carries profound significance. This study explored the comparative…

Distractor analyses are routinely conducted in educational assessments with multiple-choice items. In this research report, we focus on three item response models for distractors: (a) the traditional nominal response (NR) model, (b) a combination of a two-parameter logistic model for item scores and a NR model for selections of incorrect…

Although International Association for the Evaluation of Educational Achievement-pioneered international large-scale assessment (ILSA) of education is now a well-established science, non-practitioners and many users often substantially misunderstand how large-scale assessments are conducted, what questions and challenges they are designed to…

PISA claims that it can extend its reach from its current core subjects of Reading, Science, Maths and problem-solving. Yet given the requirement for high levels of reliability for PISA, especially in the light of its current high stakes character, proposed widening of its subject coverage cannot embrace some important aspects of the social and…

The importance of subscores in educational and psychological assessments is undeniable. Subscores yield diagnostic information that can be used for determining how each examinee's abilities/skills vary over different content domains. One of the most common criticisms about reporting and using subscores is insufficient reliability of subscores.…

Historically speaking, students were judged long before they were marked. The tradition of marking, or scoring, pieces of work students offer for assessment is little more than two centuries old, and was introduced mainly to cope with specific problems arising from the growth in the numbers graduating from universities as the industrial revolution…

This study examines student growth on mathematics and reading assessments across academic years (Spring 2006 through Spring 2009) using three different growth models: hierarchical linear model (HLM), value-added model (VAM), and student growth percentile model (SGP). Comparisons across these three growth models were conducted to investigate the…

This study used a Chinese-language version of the Index of Science Reading Awareness to explore the science reading metacognition and comprehension of Taiwanese students. Structural equation modelling results confirmed the underlying model comprised three clusters of metacognitive knowledge: beliefs and confidence in science reading, knowledge of…

The Yes/No Angoff and Bookmark method for setting standards on educational assessment are currently two of the most popular standard-setting methods. However, there is no research into the comparability of these two methods in the context of language assessment. This study compared results from the Yes/No Angoff and Bookmark methods as applied to…

Accountability for student achievement is required by legislation and demanded by the public. Testing is the method of choice for determining student achievement and for informing teachers, parents and students about what students know and still need to learn. State tests that meet the demands of federal legislation have far-reaching consequences…

In this study, the Canadian English and French versions of the Problem-Solving Measure of the Programme for International Student Assessment 2003 were examined to investigate their degree of measurement comparability at the item- and test-levels. Three methods of differential item functioning (DIF) were compared: parametric and nonparametric item…

Student achievement to reward or sanction schools. These unadjusted accountability indicators do not account for differences in student or school characteristics that contribute to variations in assessment results. Since the "Coleman Report" (1966), a guiding principle in accountability design has been that educational outcomes data…

In this research investigation of score comparability for language minority students (English language learners [ELLs] and former English language learners), we examined 3 indicators of score comparability (reliability, internal test structure, and differential item functioning) for 4th and 8th grade students who took the NCLB-mandated content…

We developed a rubric for measuring students' ability to reason and write scientifically. The Rubric for Science Writing (Rubric) was tested in a variety of undergraduate biology laboratory courses (total n = 142 laboratory reports) using science graduate students (teaching assistants) as raters. Generalisability analysis indicates that the Rubric…