Elements of arbitrariness in the standard setting process are explored, and an alternative to the use of cut scores is presented. The first part of the paper analyzes the use of cut scores in large-scale assessments, discussing three different functions: (1) cut scores define the qualifications used in assessments; (2) they simplify the reporting…

Discussed are nine questions regarding standard setting issues in educational testing: (1) Should normative or content-referenced standards be used? (2) Different standard setting methods yield different results. Does this finding present a problem? (3) Assess the adequacy of the grounding of various methods of standard setting in psychological…

Three studies assessed the psychometric characteristics of the Direct Standard Setting Method (DSSM). The Angoff technique was also used in each study. The DSSM requires judges to consider an examination 10 items at a time and determine the minimum items in that set a candidate should answer correctly to receive the credential. Nine judges set a…

Cut scores obtained for licensure tests using different judgmental methods of standard setting (holistic, test blueprint, Angoff, and modified Angoff) were compared. Nineteen educators and practitioners participated in this study as judges. Pre- and post-test feedback (feedback of total- and low-group item p-value) ratings were obtained under the…

This paper is a report of a study designed to develop recommendations on minimum qualifying scores for National Teacher Examinations (NTE) that are valid for certification and endorsement in Tennessee. The functions performed in the review of the NTE Core Battery and Specialty Area tests were conceptualized as panel activities. The number of…

Discrepancies among the standards produced by different criterion-referenced standard-setting techniques may be the result of a failure to adequately define the minimally competent candidate. Current research in this area is reviewed in terms of three categories: studies in which no formal assistance in conceptualization is given to judges,…

This paper evaluates the magnitude of the total error in estimates of the difference between an examinee's domain score and the cutoff score. An observed score based on a random sample of items from the domain, and an estimated cutoff score derived from a judgmental standard setting procedure are assumed. The work of Brennan and Lockwood (1980) is…

This report investigates an area of uncertainty in using the Angoff method for setting standards, namely whether or not a judge's conceptualizations of borderline group performance are realistic. Ratings are usually made with reference to the performance of this hypothetical group, therefore the Angoff method's success is dependent on this point.…

In the specific methods of standard setting in testing, judgments about individuals being tested, contrasting groups being tested, and judgments about the test items are discussed. In judgments about individual test-takers, assumptions are presented based on the knowledge and skills the test is intended to measure, the test-takers' skills at the…

Test development procedures and six methods for determining cut-off scores are briefly described. An alternate method, appropriate when the test developer also determines the cut-off score, is suggested. Unlike other methods, the standard is set during the test development stage. Its computations are intelligible to nonstatistically-oriented…

The purpose of this report is to help in the evolution of these systems by examining the measures used, including, but not limited to, tests. The author asks: Are these the best measures? Are they used right? Are there other measures that should be employed? It is the model of reform itself that is examined, and the report does not address…

In the teaching profession, the National Council for Accreditation of Teacher Education (NCATE) is the professional accrediting organization for schools, colleges, and departments of education in the United States. This brochure provides a glimpse of NCATE's standards, focus, and effect on teacher quality and student learning. It explains what…

In the early 1990s, states began establishing standards in academic subjects backed by test-based accountability systems to see that the standards were met. Incentives were implemented for schools and students based on pupil test scores. These early accountability systems paved the way for passage of landmark federal legislation, the No Child Left…

U.S. student scores in science compare unfavorably with those of other nations, and other standardized test scores by U.S. students are also comparatively lower. Polls of parents indicate dissatisfaction with U.S. education. College teachers and employers feel that high school graduates are weak in skills. This document offers negative perceptions…

Over the past two decades, America has embarked on a campaign to improve public schools for all our nation's children. Educators have turned their focus to the outcomes of schooling. The Committee for Economic Development (CED) has urged such reforms, and this report concentrates on the need to link the way schools are funded and managed to the…