The cognitive components standard setting method, recently introduced by D. McGinty and J. Neel (1996), asks judges to specify minimum levels of performance not for the test items, but for smaller portions of items, the component skills and concepts required to answer each item correctly. Items are decomposed into these components before judges…

Many school districts are developing assessments that incorporate both selected response and constructed response formats. Scores on these assessments can be used for a variety of purposes ranging from subject remediation to promotion decisions. These policy decisions are informed by recommendations for Minimum Passing Scores (MPSs) from standard…

It was hypothesized that, when compared to the Angoff method (W. H. Angoff, 1971), the Nedelsky method (L. Nedelsky, 1954) for standard setting had lower intrajudge inconsistency, lower cutscores, and lower cutscores especially for items presenting challenges to the judges. These hypotheses were tested and supported in a sample of 22 graduate…

The procedures projected to be used in the spring and summer of 1982 to establish passing standards for the Maryland Functional Mathematics Test are described. The development of the test as a prerequisite for high school graduation to be implemented in 1987 is described. The maximum involvement of diverse constituencies and multiple informed…

The primary objective of this research was to examine the effect of scoring items known to be multidimensional using a unidimensional model. Although several simulation studies have examined this, few studies have been conducted using data obtained from actual test administrations. In this study, open-ended mathematics items from a mandated state…

Studies of the Angoff method of standard setting suggest that judges agree in their estimates of the relative difficulties of test questions for minimally competent examinees and that each judge's estimates correlate well with the observed item difficulties for examinees whose total test scores are near the judge's personal standard (G. E.…

In 1991 the Measurement and Evaluation Center of the University of Texas at Austin was asked to develop a test for credit by examination in four lower division courses in Japanese. The test (in Japanese) was constructed from locally developed items provided by instructors of Japanese. The developed test consisted of 80 items distributed among…

Setting standards on performance measures is discussed in the context of the State Collaborative on Assessment and Student Standards (SCASS) initiative supported by the Council of Chief State School Offices. The usual item-based methods for standard setting, the methods developed by Nedelsky (1954), Angoff (1971), and Ebel (1972), were developed…

The effects of feedback about the ratings of other judges on subsequent ratings using the Nedelsky method and the ability of judges to retain or eliminate options in a manner consistent with the judgments of minimally competent examinees were studied using data from a basic algebra examination administered to 227 college students in 1987. The…

The research reported in this paper was conducted to gain information to guide the selection of standard setting procedures for use with polytomous items to set achievement levels on the National Assessment of Educational Progress (NAEP) assessments in U.S. History and geography. Standard-setting procedures were evaluated to determine the relative…

The purpose of the NMSBA technical report is to provide users and other interested parties with a general overview of and technical characteristics of the 2007 NMSBA. The 2007 technical report contains the following information: (1) Test development; (2) Scoring procedures; (3) Summary of student performance; (4) Statistical analyses of item and…

Techniques such as the norm-referenced and average score techniques, commonly used in the identification of educationally disadvantaged students, are critiqued. This study applied latent trait theory, specifically the Rasch Model, along with teacher judgments relative to the mastery of instructional/test decisions, to derive a standard setting…

There can be little doubt that there is wide variation in the rigor and quality of state standards and assessments. Moreover, it is clear that the vast majority of states have set their academic achievement bar far lower than federal standards, as measured by the National Assessment of Educational Progress (NAEP). Since No Child Left Behind (NCLB)…

Minimum competency cut-off scores on a statistics exam were estimated under four conditions: the Angoff judging method with item data (n=20), and without data available (n=19); and the Modified Angoff method with (n=19), and without (n=19) item data available to judges. The Angoff method required free response percentage estimates (0-100) percent,…

Studies of the mathematics assessments of the National Assessment of Educational Progress (NAEP) are summarized. One study found that omit rates for NAEP test items were higher for African Americans and Hispanics than for whites. The other found that descriptions and examples for the 1992 mathematics achievement levels were misleading. (SLD)