First-year engineering (FYE) students are routinely screened for spatial ability, with the goals of predicting retention in the major and identifying those who need supplementary spatial instruction. Psychometric tests used for such screenings are often domain-general measures that lack diagnostic information to inform remedial instruction. A new…

The basis for understanding neurophysiology is understanding ion movement across cell membranes. Students in introductory courses recognize ion concentration gradients as a driving force for ion movement but struggle to simultaneously account for electrical charge gradients. We developed a 17-multiple-choice item assessment of students'…

Digital information literacy (DIL)--generally defined as the ability to obtain, understand, evaluate, and use information in a variety of digital technology contexts--is a critically important skill deemed necessary for success in higher education as well as in the global networked economy. To determine whether college graduates possess the…

Plausible distractors are important for accurate measurement of knowledge via multiple-choice questions (MCQs). This study demonstrates the impact of higher distractor functioning on validity and reliability of scores obtained on MCQs. Freeresponse (FR) and MCQ versions of a neurohistology practice exam were given to four cohorts of Year 1 medical…

Student's ability to use and manipulate scientific concepts has been widely explored; however there is still a need to define the characteristics and nature of science ability. Also, the tests and performance scales that require minimal conceptual knowledge to measure this ability are relatively less common. The aim of this study was to develop an…

The Organisation for Economic Co-operation and Development (OECD) launched the Assessment of Higher Education Learning Outcomes (AHELO) in an effort to measure learning in international postsecondary education. This paper presents a study of scoring equivalence across nine countries for two translated and adapted performance tasks. Results reveal…

Despite three decades of advances in information and communications technology (ICT) and a generation of research on cognition and new pedagogical strategies, the field of assessment has not progressed much beyond paper-and-pencil item-based tests. Research has shown these instruments are not valid measures of sophisticated intellectual…

Significant momentum exists for next-generation assessments to increasingly utilize technology to develop and deliver performance-based assessments. Many traditional challenges with this assessment approach still apply, including psychometric concerns related to performance-based tasks (PBTs), which include low reliability, efficiency of…

Although the belief has been expressed that performance assessments are intrinsically more fair than multiple-choice measures, some forms of performance assessment may in fact be more likely than conventional tests to tap construct-irrelevant factors. As performance assessment grows in popularity, it will be increasingly important to monitor the…

Examination of the role of psychometrics in the development of multiple-choice tests and performance-based assessments and consideration of validity and reliability issues leads to this conclusion: Choice of assessments for instruction or large-scale accountability depends on which is more appropriate for the particular purposes. Taxonomic…

This rejoinder to Grant Wiggins on performance assessment suggests that true educational reform will undoubtedly be evidenced by something more substantial than pocket folders bulging with student work. Labeling performance tests "authentic" does not ensure their validity, reliability, or incorruptibility. Such tests are neither replacements nor…

This study examines differences and similarities in the information provided by direct and indirect measures of writing from the Collegiate Assessment of Academic Proficiency (CAAP). The indirect measure was a 72-item multiple-choice test, while the direct measure involved responding to two essay prompts. The 40-minute multiple-choice test can be…

Describes a case study to determine whether performance tests are a feasible alternative to multiple-choice tests. Examines the difficulties of administering and scoring performance assessments. Explains that the study employed three performance tests and one multiple-choice test. Concludes that performance test administration and scoring was no…

Describes recent efforts of the Center for Research in Evaluation, Standards and Student Testing (CRESST) to evaluate authentic assessment methods, such as portfolios and performance tests. When comparing the merits of authentic versus multiple-choice testing, it is wise to consider validity, reliability, consequences, fairness, generalization,…

Discusses how recent court decisions and the move toward performance assessment may affect the adverse impact, reliability, validity, and pass-fail standards of teacher-certification tests. Recommendations are made for tests that combine multiple-choice items with open-ended tasks. (SLD)