There is always a chance that examinees will answer multiple choice (MC) items correctly by guessing. Design choices in some modern exams have created situations where guessing at random through the full exam--rather than only for a subset of items where the examinee does not know the answer--can be an effective strategy to pass the exam. This…

The overclaiming technique is a novel assessment procedure that uses signal detection analysis to generate indices of knowledge accuracy (OC-accuracy) and self-enhancement (OC-bias). The technique has previously shown robustness over varied knowledge domains as well as low reactivity across administration contexts. Here we compared the OC-accuracy…

Although the belief has been expressed that performance assessments are intrinsically more fair than multiple-choice measures, some forms of performance assessment may in fact be more likely than conventional tests to tap construct-irrelevant factors. As performance assessment grows in popularity, it will be increasingly important to monitor the…

After an extensive review of the available research on large-scale writing assessment, certain issues in writing assessment seem to be unresolved, and still other issues are not supported by adequate research. This paper reviews the basic issues in writing assessment, points out which topics are supported by strong research, and which topics are…

A multiple-choice test was developed to assess cognitive gains resulting from participation in the School Health Education Curriculum Project. The test was intended for seventh grade students, and was based on curricular objectives in the cognitive domain: knowledge, comprehension, application, and evaluation. Educators who were familiar with the…

This report describes a methodology for increasing the validity and reliability of inferences made about the problem-solving ability of science students that is based on performance on different kinds of tests. The generalizable cognitive components of problem solving that might be targeted by assessment are described, and specifications are…

Alternatives to multiple-choice tests for teacher licensing examinations are described, and their advantages are cited. Concerns are expressed in the areas of cost and practicality, reliability, corruptibility, and validity. A suggestion for reducing costs using multiple-choice responses calibrated to constructed-response tasks is proposed. (SLD)

Operating from multiple literature bases in cognitive psychology, mathematics education, and theoretical and applied psychometrics, Schilling, Hill and their colleagues provide a systemic approach to studying the validity of scores of mathematical knowledge for teaching. This system encompasses an array of task formats and methodologies. The…

This paper summarizes an investigation of applications and issues in free response (FR) testing during 1989. It draws on ideas from the results of the National Educational Longitudinal Study 1988 (NELS:88) field test, a seminar series at the Educational Testing Service (ETS), working papers prepared for several FR testing applications, and…

A review of the recent literature has yielded a number of concerns about the validity, reliability, cost, efficiency, generalizability, utility, and cultural sensitivity of performance based assessments. The resulting conclusion was that continuing the performance based assessment initiative should be rethought. Suggested alternatives included…

Issues in designing valid tests for the National Assessment of Educational Progress (NAEP) are discussed. Test scores are often provided without any information on the nature of the tasks represented. Because test domains are defined by individual item writers, the generalizability between tests and items is suspect. While typical content…

This paper provides a summary of the authors' attempts to uncover links between their measures, classroom mathematics instruction, and student learning. This paper also provides evidence regarding one central critique of their measures: that multiple-choice assessments cannot validly represent the knowledge, skills, and judgment involved in actual…

Discusses the movement toward authentic assessment, also called direct or performance assessment, as an alternative to multiple-choice, standardized, norm-referenced testing. Authentic testing involves assessment tasks that are real instances of extended criterion performances rather than proxies of actual learning goals. Questions use of…

Writing encompasses a wide range of skills, from the mechanics of punctuation and spelling to the systematic or even creative development of ideas. The higher order skills of communication necessarily involve critical thinking and problem solving, and an assessment of the writing of advanced students will be able to encompass these other higher…

Following a practical approach, this book focuses on educational assessment as it is practiced today, in the actual world of classrooms, school systems, state departments, and national organizations. It first covers the basic concepts applicable to all types of assessment: statistical concepts, reliability, validity, and interpretive frameworks.…