A critical component of the standard setting process is collecting evidence to evaluate the recommended cut scores and their use for making decisions and classifying students based on test performance. Kane (1994, 2001) proposed a framework by which practitioners can identify and evaluate evidence of the results of the standard setting from (1)…

Recently, Northwest Evaluation Association (NWEA) completed a study to connect the scale of the Minnesota Comprehensive Assessments (MCA) Testing Program used for Minnesota's mathematics and reading assessments with NWEA's RIT (Rasch Unit) scale. Information from the state assessments was used in a study to establish performance-level scores on…

Recently, the Northwest Evaluation Association (NWEA) completed a study to connect the scale of the North Carolina State End of Grade (EOG) Testing Program used for North Carolina's mathematics and reading assessments with NWEA's Rausch Interval Unit (RIT) scale. Information from the state assessments was used in a study to establish…

Standard setting is a method used to set cut scores on large-scale assessments. One of the most popular standard setting methods is the Bookmark method. In the Bookmark method, panelists are asked to envision a response probability (RP) criterion and move through a booklet of ordered items based on a RP criterion. This study investigates whether…

In this article we address the issue of consistency in standard setting in the context of an augmented state testing program. Information gained from the external NRT scores is used to help make an informed decision on the determination of cut scores on the state test. The consistency of cut scores on the CRT across grades is maintained by forcing…

The bookmark standard-setting procedure is an item response theory-based method that is widely implemented in state testing programs. This study estimates standard errors for cut scores resulting from bookmark standard settings under a generalizability theory model and investigates the effects of different universes of generalization and error…

Psychometricians continue to introduce new approaches to setting cut scores for educational assessments in an attempt to improve on current methods. In this paper we describe the Item-Descriptor (ID) Matching method, a method based on IRT item mapping. In ID Matching, test content area experts match items (i.e., their judgments about the knowledge…

Many approaches to standard-setting use item calibration and student score estimation results to structure panelists' tasks. However, this requires collecting standard-setting judgments after the item analysis results are available. The Scoring Guide Alignment approach collects standard-setting judgments during the scoring sessions from teachers…

This document contains the results of a standard setting conducted in January 2002 on the Delaware Student Testing Program (DSTP) Science and Social Studies tests at grades 4 and 6. Each standard setting process entailed convening four groups, one for each grade level and content area, and each group met for 2 days. At the standard setting judges…

This Brief builds on the work of the National Association of State Boards of Education 1997 Study Group on State Assessment Systems by examining one of the state board actions that is most likely to capture the publics attention: setting cut scores on state assessments. This process involves a large measure of human judgment and politics and a…

Outlines a framework for considering the validity of standards-based score interpretations and then considers the potential roles of different stakeholder groups and other participants in that process. Suggests study of a new standard-setting method, the "briefing book," which would describe alternative cut scores. (SLD)

In a coherent educational assessment system, the relation between the cut scores across the grades of a content area should reflect the goals of the educational system moderated by the state of the standards within that system. In this article, we discuss the need for consistent cut scores across the grades of a content area; we present and…

This paper explores common concerns about competency testing as they relate to the certification of high school graduates seeking a diploma in the United States. Competency testing is widespread in the United States, with 40 states engaged in competency testing in at least one grade. In general, and particularly for graduation requirements, the…

Standard setting practices in states using the NTE (formerly the National Teacher Examinations) were examined for 1987. The NTE is composed of two segments: (1) a Core Battery covering the communication skills and general knowledge skills; and (2) a test of professional knowledge about teaching. The processes used to establish passing scores for…

In the context of controversy over the use of high stakes testing, the Pennsylvania State Education Association (PSEA) asked for an evaluation of the performance levels and cut score of the Pennsylvania System of School Assessment (PSSA) mathematics and reading tests. While awaiting technical documentation from the Pennsylvania Department of…