Distractor analyses are routinely conducted in educational assessments with multiple-choice items. In this research report, we focus on three item response models for distractors: (a) the traditional nominal response (NR) model, (b) a combination of a two-parameter logistic model for item scores and a NR model for selections of incorrect…

We present a logistic function of a monotonic polynomial with a lower asymptote, allowing additional flexibility beyond the three-parameter logistic model. We develop a maximum marginal likelihood-based approach to estimate the item parameters. The new item response model is demonstrated on math assessment data from a state, and a computationally…

One major issue with large-scale assessments is that the respondents might give no responses to many items, resulting in less accurate estimations of both assessed abilities and item parameters. This report studies how the types of items affect the item-level nonresponse rates and how different methods of treating item-level nonresponses have an…

The overclaiming technique is a novel assessment procedure that uses signal detection analysis to generate indices of knowledge accuracy (OC-accuracy) and self-enhancement (OC-bias). The technique has previously shown robustness over varied knowledge domains as well as low reactivity across administration contexts. Here we compared the OC-accuracy…

The purpose of this quantitative study is to compare a new assessment tool, the SkillsUSA Connect Assessment with the NOCTI assessment to determine which test results in more students achieving success. A quantitative study, designed to compare test scores of students taking the NOCTI assessment and new assessments from SkillsUSA, called the…

The Yes/No Angoff and Bookmark method for setting standards on educational assessment are currently two of the most popular standard-setting methods. However, there is no research into the comparability of these two methods in the context of language assessment. This study compared results from the Yes/No Angoff and Bookmark methods as applied to…

For educational achievement tests that employ multiple-choice (MC) items and aim to reliably classify students into performance categories, it is critical to design MC items that are capable of discriminating student performance according to the stated achievement levels. This is accomplished, in part, by clearly understanding how item design…

This work compares the results of three assessment systems used in two Spanish universities (the "Universidad Politecnica de Madrid" and the "Universidad Catolica de Avila"): the traditional system based on final examinations, continuous assessment with periodic tests and a proposed system (specially designed for heterogeneous…

Describes a study of the accuracy of student responses on objective tests. Investigators examined the frequency of correctness on initial responses versus changed responses, and the relationship to degree of familiarity of the content. Results show that changing test answers tends to produce more right than wrong answers by more students.…

Connecticut's experience with four different standard-setting methods regarding multiple choice proficiency tests is described. The methods include Angoff, Nedelsky, Borderline Group, and Contrasting Groups Methods. All Connecticut ninth graders were administered proficiency tests in reading, language arts, and mathematics. As soon as final test…

To investigate the ability of multiple-choice tests to assess higher order thinking skills, examinations were constructed as half multiple choice and half constructed response. Results with 90 undergraduate and graduate students indicate that the 2 formats measure similar constructs at different levels of complexity. (SLD)

Many state and local entities are developing and using performance assessment programs. Because these initiatives are so diverse, it is very difficult to understand what they are doing, or to compare them in any meaningful way. Multiple-choice tests are contrasted with performance assessments, and preliminary classifications are suggested to…

This paper summarizes an investigation of applications and issues in free response (FR) testing during 1989. It draws on ideas from the results of the National Educational Longitudinal Study 1988 (NELS:88) field test, a seminar series at the Educational Testing Service (ETS), working papers prepared for several FR testing applications, and…

The resources necessary to create, administer, and score performance assessments in science were studied. RAND and the University of California, Santa Barbara (UCSB) designed performance tasks for science in grades five and six as part of a larger study of the feasibility of science performance assessment. Tasks were developed in pairs in task…

Problems with performance assessment (PA) and multiple-choice tests (MCTs) are outlined, with reference to the literature on accountability. PA for individual teachers who should integrate their assessments with their instruction; PA as a supplement to more traditional examinations for licensure decisions; and some limited, experimental tryouts of…