Optimal assessment tools should measure in a limited time the knowledge of students in a correct and unbiased way. A method for automating the scoring is multiple choice scoring. This article compares scoring methods from a probabilistic point of view by modelling the probability to pass: the number right scoring, the initial correction (IC) and…

Concordance tables have been used for decades to relate scores on different tests measuring similar but distinct constructs. These tables, typically derived from statistical linking procedures, provide a direct link between scores on different tests and serve various purposes. Aside from describing how a score on one test relates to performance on…

This study examined students' competencies in engineering education at the university level. First, we developed a competency model in one specific field of engineering: process dynamics and control. Then, the theoretical model was used as a frame to construct test items to measure students' competencies comprehensively. In the empirical…

Student responses to the West Virginia Educational Standards Test (WESTEST) 2 Online Writing Assessment are scored by a computer-scoring engine. The scoring method is not widely understood among educators, and there exists a misperception that it is not comparable to hand scoring. To address these issues, the West Virginia Department of Education…

The New York State Board of Regents has committed to the transformation of the preparation, support and evaluation of all teachers and school leaders in New York State, and the New York State Legislature has enacted historic legislation (Education Law §3012-c) that fundamentally changes the way teachers and principals are evaluated. Under the new…

AIR is working with states and districts across the country to improve teacher evaluation and feedback. Our work is focused on designing systems of educator evaluation and compensation that incorporate multiple measures of performance and, in particular, measures of student growth. In this work, student learning objectives (SLOs) have emerged as a…

Let X be a statistical variable representing student ratings of University teaching. It is natural to assume for X an ordinal scale consisting of k categories (in ascending order of satisfaction). At first glance, student ratings can be summarized by a location index (such as the mode or the median of X) associated with a convenient measure of…

This study assessed the invariance in the factor structure of the "Test of English as a Foreign Language"™ Internet-based test (TOEFL® iBT) across subgroups of test takers who differed in native language and exposure to the English language. The subgroups were defined by (a) Indo-European and Non-Indo-European language family, (b)…