Large-scale performance assessment programs are a longstanding reform tool. However, standard setting can be a challenge for assessment programs that use primarily non-standardized assessments. The purpose of this paper is to extend this field of research by explaining the standard setting methodology applied to one more recent instantiation of a…

Students in a junior- and senior-level Anatomy and Physiology course have the opportunity to correct missed exam questions ("regrade") and earn up to half of the original points missed. The three objectives of this study were to determine if: (a) performance on the regrade assignment was correlated with scores on subsequent exams, (b)…

Assessments represent an important component of undergraduate courses because they affect how students interact with course content and gauge student achievement of course objectives. To make decisions on assessment design, instructors must understand the affordances and limitations of available question formats. Here, we use a crossover…

As a property of test scores, reliability/dependability constitutes an important psychometric consideration, and it underpins the validity of measurement results. A review of interpreter certification performance tests (ICPTs) reveals that (a) although reliability/dependability checking has been recognized as an important concern, its theoretical…

We use a natural experiment to show that the presence of an external examiner has both a direct and an indirect negative effect on the performance of monitored classes in standardized educational tests. The direct effect is the difference in the test performance between classes of the same school with and without external examiners. The indirect…

The Mayo-Portland Adaptability Inventory 4 (MPAI-4) is a valid and reliable assessment tool to detect clinical impairments in patients with acquired brain injury. The tool is widely used by rehabilitation therapists worldwide, given its good psychometric properties and its availability in several languages. The purpose of this study was to…

The present study examined the effect accommodations have on test results of students with and without disabilities and documented experts' judgments about the appropriateness of testing accommodations. Test score data were collected from 218 fourth-grade students with and without disabilities on mathematics and science performance tasks and from…

Education policy in the U.S. in the last two decades has emphasized large-scale assessment of students, with growing consequences for schools, teachers, and students. Given the high stakes of such tests, it is important to understand the relationships between students' answers to test items and their knowledge and skills in the tested content…

A key component to the new wave of performance-based pay initiatives is the use of student achievement data to evaluate teacher performance. As greater amounts of student achievement data are being collected, researchers have been developing and applying innovative statistical and econometric models to longitudinal data to develop measures of an…

Through the standard error, rather than a reliability coefficient, generalizability theory provides an indicator of the uncertainty attached to school and individual scores on performance assessments. Recommendations are made to apply generalizability theory to current performance assessments, emphasizing practices that differ from usual…