The object of research is a statistics exam that contains problem tasks. One examiner performed two exam evaluation methods to repeatedly evaluate the exam. The goal was to compare the methods for objectivity. One of the two exam evaluation methods we call a serial evaluation method. The serial evaluation method assumes evaluation of all exam…

Providing evidence that can inform awarding is an important application of Comparative Judgement (CJ) methods in high-stakes qualifications. The process of marking scripts is not changed, but CJ methods can assist in the maintenance of standards from one series to another by informing decisions about where to place grade boundaries or cut scores.…

Tests of statistical significance often play a decisive role in establishing the empirical warrant of evidence-based research in education. The results from pattern-based assessment items, as introduced in this paper, are categorical and multimodal and do not immediately support the use of measures of central tendency as typically related to…

We investigated French primary school teachers' assessment practice in mathematics. Using an online questionnaire on teachers' background, teaching, and grading practice, we were able to determine assessment profiles of 604 primary school teachers. As evidenced by the teachers' scores on the latent factors Assessment purposes, Assessment…

In educational and psychological research, it is common to use latent factors to represent constructs and then to examine covariate effects on these latent factors. Using empirical data, this study applied three approaches to covariate effects on latent factors: the multiple-indicator multiple-cause (MIMIC) approach, multiple group confirmatory…

The purpose of this study was to conduct a conceptual replication of Pendergast et al.'s (2018) study that examined the diagnostic accuracy of a nomogram procedure, also known as a naive Bayesian approach. The specific naive Bayesian approach combined academic and social-emotional and behavioral (SEB) screening data to predict student performance…

In this analysis, we examine how course assessment items were aligned with learning objectives in a quantitative literacy course at Michigan State University. The alignment analysis consisted of mapping assessment items to a list of operationalized learning objectives from the course. Our analysis shows how often the learning objectives are…

Observation systems are increasingly used around the world for a variety of purposes; 2 critical purposes are to understand and to improve teaching. As observation systems differ considerably, individuals must decide what observation system to use. But the field does not have a common specification of an observation system, nor does it have…

A variety of differential item functioning (DIF) methods have been proposed and used for ensuring that a test is fair to all test takers in a target population in the situations of, for example, a test being translated to other languages. However, once a method flags an item as DIF, it is difficult to conclude that the grouping variable (e.g.,…

Prerequisite skill structures have been closely studied in past years leading to many data-intensive methods aimed at refining such structures. While many of these proposed methods have yielded success, defining and refining hierarchies of skill relationships are often difficult tasks. The relationship between skills in a graph could either be…

The purpose of this research was to develop observed score and true score equating procedures to be used in conjunction with the multidimensional item response theory (MIRT) framework. Three equating procedures--two observed score procedures and one true score procedure--were created and described in detail. One observed score procedure was…

Considering the amount of funding that is distributed to educational research each year, leaders and policymakers have a vested interest in finding scientifically based evidence that answers causal questions regarding program effectiveness. The importance of program evaluation has long been recognized in many fields of research; however, the most…

Trend estimation in international comparative large-scale assessments relies on measurement invariance between countries. However, cross-national differential item functioning (DIF) has been repeatedly documented. We ran a simulation study using national item parameters, which required trends to be computed separately for each country, to compare…

The purpose of the present study was to investigate whether a specific testing accommodation (extended time) affects test scores for college students with and without ADHD. College students with ADHD (N = 61) and without ADHD (N = 68) took a math test, after having been told they had either standard time or extended time to complete the test.…

This report reviews the major international and regional large-scale educational assessments, including international surveys, school-based surveys and household-based surveys. The report compares and contrasts the cognitive and contextual data collection instruments and implementation methods used by the different assessments in order to identify…