Cut scores obtained for licensure tests using different judgmental methods of standard setting (holistic, test blueprint, Angoff, and modified Angoff) were compared. Nineteen educators and practitioners participated in this study as judges. Pre- and post-test feedback (feedback of total- and low-group item p-value) ratings were obtained under the…

This paper is a report of a study designed to develop recommendations on minimum qualifying scores for National Teacher Examinations (NTE) that are valid for certification and endorsement in Tennessee. The functions performed in the review of the NTE Core Battery and Specialty Area tests were conceptualized as panel activities. The number of…

Discrepancies among the standards produced by different criterion-referenced standard-setting techniques may be the result of a failure to adequately define the minimally competent candidate. Current research in this area is reviewed in terms of three categories: studies in which no formal assistance in conceptualization is given to judges,…

This paper evaluates the magnitude of the total error in estimates of the difference between an examinee's domain score and the cutoff score. An observed score based on a random sample of items from the domain, and an estimated cutoff score derived from a judgmental standard setting procedure are assumed. The work of Brennan and Lockwood (1980) is…

This report investigates an area of uncertainty in using the Angoff method for setting standards, namely whether or not a judge's conceptualizations of borderline group performance are realistic. Ratings are usually made with reference to the performance of this hypothetical group, therefore the Angoff method's success is dependent on this point.…

In the specific methods of standard setting in testing, judgments about individuals being tested, contrasting groups being tested, and judgments about the test items are discussed. In judgments about individual test-takers, assumptions are presented based on the knowledge and skills the test is intended to measure, the test-takers' skills at the…

Test development procedures and six methods for determining cut-off scores are briefly described. An alternate method, appropriate when the test developer also determines the cut-off score, is suggested. Unlike other methods, the standard is set during the test development stage. Its computations are intelligible to nonstatistically-oriented…

The purpose of this report is to help in the evolution of these systems by examining the measures used, including, but not limited to, tests. The author asks: Are these the best measures? Are they used right? Are there other measures that should be employed? It is the model of reform itself that is examined, and the report does not address…

In the teaching profession, the National Council for Accreditation of Teacher Education (NCATE) is the professional accrediting organization for schools, colleges, and departments of education in the United States. This brochure provides a glimpse of NCATE's standards, focus, and effect on teacher quality and student learning. It explains what…

In the early 1990s, states began establishing standards in academic subjects backed by test-based accountability systems to see that the standards were met. Incentives were implemented for schools and students based on pupil test scores. These early accountability systems paved the way for passage of landmark federal legislation, the No Child Left…

U.S. student scores in science compare unfavorably with those of other nations, and other standardized test scores by U.S. students are also comparatively lower. Polls of parents indicate dissatisfaction with U.S. education. College teachers and employers feel that high school graduates are weak in skills. This document offers negative perceptions…

Over the past two decades, America has embarked on a campaign to improve public schools for all our nation's children. Educators have turned their focus to the outcomes of schooling. The Committee for Economic Development (CED) has urged such reforms, and this report concentrates on the need to link the way schools are funded and managed to the…

A study was conducted to explore empirically the effect of different definitions of the target examinee on the judgment of panelists setting a passing score. Two cut score studies were done in a school district for the same test within a 6-month period, and different definitions of the target candidate were provided for each study. In October…

The Educational Policy Reform Research Institute (EPRRI) and the US Department of Education's Office of Special Education Programs (OSEP) held a policy symposium entitled "Ensuring Accountability for All Children in an Era of Standards-Based Reform: Alternate Achievement Standards" February 4-6, 2004 at the Crystal City Marriott in Arlington,…

Following a review of educational standard-setting experiences in the United States, the procedures used by the National Assessment of Educational Progress (NAEP) to arrive at a consensus on educational objectives are described. These objectives represent the only definitions of educational quality that are widely accepted by professional and lay…