There is always a chance that examinees will answer multiple choice (MC) items correctly by guessing. Design choices in some modern exams have created situations where guessing at random through the full exam--rather than only for a subset of items where the examinee does not know the answer--can be an effective strategy to pass the exam. This…

This technical report provides an overview of the New York State Alternate Assessment (NYSAA), including a description of the purpose of the NYSAA, the processes utilized to develop and implement the NYSAA program, and Stakeholder involvement in those processes. The purpose of this report is to document the technical aspects of the 2013-14 NYSAA.…

Although the Angoff procedure is among the most widely used standard setting procedures for tests comprising multiple-choice items, research has shown that subject matter experts have considerable difficulty accurately making the required judgments in the absence of examinee performance data. Some authors have viewed the need to provide…

Elements of arbitrariness in the standard setting process are explored, and an alternative to the use of cut scores is presented. The first part of the paper analyzes the use of cut scores in large-scale assessments, discussing three different functions: (1) cut scores define the qualifications used in assessments; (2) they simplify the reporting…

In 1991 the National Assessment Governing Board (NAGB) released a report interpreting the achievement of U.S. students in mathematics on the 1990 National Assessment of Educational Progress in terms of a set of performance standards. The NAGB had been designing and implementing an approach to defining basic, proficient, and advanced levels of…

This 1982 supplement is a further elaboration to the Writing Assessment which was distributed in 1981 to assist schools in assessment of writing by detailing scoring procedures, standard-setting, and the reporting of results. The supplement deals with retesting, alternative methods for setting standards, establishing score equivalency on writing…

Comparative results are presented for procedures recently appearing in literature related to standard setting on the National Assessment of Educational Progress--the paper selection method and the contrasting group method. For this comparison, a probabilistic model with normal distribution of performance and a six-point scale were assumed. The…

The number of performance-based assessments is increasing rapidly, but to date there is no established procedure for setting standards on these assessments. This paper describes several extensions to the Angoff procedure to accommodate the characteristics of a performance-based assessment and presents the results of research in applying this…

As constructed-response items become an integral part of educational assessments, setting student performance standards on constructed-response items has become an important issue. Two standard-setting methods, one used for setting standards on the National Assessment of Educational Progress (NAEP) in reading in grade 8 and the other used to set…

This paper reports 20 unsolved problems in educational measurement and points toward what seem to be promising avenues of solution. The first group of concerns involves validity, posing the problems of obtaining suitable validity criteria, determining and measuring the predictor variables that best characterize the traits and proficiencies of…

The New Standards Project is a partnership of 17 states and 6 major school districts that have joined to develop alternative approaches to setting educational standards and assessing student achievement. In the New Standards Project plan, formal standard setting occurs through a process of public examination of student work products on pilot…

A scoring rubric transition study is described. It was designed to evaluate possible drift in scoring the Maryland Writing Test from year to year (when using a modified holistic scoring method), to evaluate strategies for revising swing rubrics from narrative and explanatory writing while maintaining original scoring standards, and to establish…

The most important issue in education today is helping students reach high standards. These standards are changing American education from a system driven by inputs and regulations to one judged by results. This manual is intended for use with any set of standards. Followed step by step, it will take the teacher from the abstract statements of the…

Praxis III is an assessment procedure that provides information for making instructional and licensing decisions about beginning teachers. The Praxis III Assessor's job is to interview the beginning teacher, observe the teacher in the classroom, score the teacher's performance on 19 criteria, and summarize the evidence for each score. The Assessor…

Setting standards on performance measures is discussed in the context of the State Collaborative on Assessment and Student Standards (SCASS) initiative supported by the Council of Chief State School Offices. The usual item-based methods for standard setting, the methods developed by Nedelsky (1954), Angoff (1971), and Ebel (1972), were developed…