In this paper we assert that the practice of principled assessment design renders traditional standard-setting methodology redundant at best and contradictory at worst. We describe the rationale for, and methodological details of, Embedded Standard Setting (ESS; previously, Engineered Cut Scores. Lewis, 2016), an approach to establish performance…

A task force was created in a small, AACSB-accredited business school to develop a more comprehensive set of standards for faculty performance. The task force relied heavily on faculty input to identify and describe key dimensions that capture effective teaching and service performance. The result is a multi-dimensional framework that will be used…

Recently, there has been a significant expansion in the use of performance assessment in large scale testing programs. Although there has been significant support from curriculum and policy stakeholders, the technical feasibility of large scale performance assessments has remained a question. This report is intended to contribute to the debate by…

A newly developed performance standard-setting procedure, termed iterative judgmental policy capturing (JPC), is applicable to assessments composed of distinct multidimensional exercises. The procedure is described, and results are reported from the application of JPC in a study involving a panel of 20 teachers and 6 performance exercises. (SLD)

Reviews two models for assessment, the measurement model and the standards model, their underlying assumptions about learners, and the resulting implications for performance-based test development. Discusses the current testing debate, defines terms such as "authentic assessment" and "performance-based assessment," and…

Development of a multistage dominant profile method for setting standards on complex performance assessments is detailed. The method grew from experiences with a judgmental policy-capturing procedure and an extended Angoff method. The design of an early adolescence English language arts assessment illustrates the complexity of decisions panelists…

The number of performance-based assessments is increasing rapidly, but to date there is no established procedure for setting standards on these assessments. This paper describes several extensions to the Angoff procedure to accommodate the characteristics of a performance-based assessment and presents the results of research in applying this…

A study was conducted to provide information for setting two separate standards, the accuracy score and the documentation score, for the Praxis III: Classroom Performance Assessment (Praxis III). Praxis III is intended for making instructional and licensing decisions about beginning teachers. This standard-setting study was a person-judgment…

The International Language Testing Association has some 250 members in 15 countries. Most are specialists in the testing of second language skills, with a special interest in performance assessment because of the testing of speaking and writing performance that is critical to second language skills assessment. The association believes that certain…

The most important issue in education today is helping students reach high standards. These standards are changing American education from a system driven by inputs and regulations to one judged by results. This manual is intended for use with any set of standards. Followed step by step, it will take the teacher from the abstract statements of the…

Performance assessments in education and credentialing are becoming popular. At the same time, there do not exist any well established and validated methods for setting standards on performance assessments. This paper describes several of the new standard-setting methods that are emerging for use with performance assessments and considers their…

Setting standards on performance measures is discussed in the context of the State Collaborative on Assessment and Student Standards (SCASS) initiative supported by the Council of Chief State School Offices. The usual item-based methods for standard setting, the methods developed by Nedelsky (1954), Angoff (1971), and Ebel (1972), were developed…

The accountability testing programs of many states have begun to make extensive use of constructed-response questions and to report test results in terms of percentages of students at various proficiency or performance levels. This paper describes the step-by-step procedures for two standard-setting methods recently used for the New Hampshire…

This research report is a summary of the information the Council of Great City Schools gathered in a survey of members about the development of standards and assessments in 1995. Some 77% (36) of the 47 member districts responded to the survey, which asked about the impetus for standards and assessment development, the areas in which they are…

Claims that national tests will improve student learning are explored, asking whether national examinations will ensure high-quality instruction and greater student learning and whether tests developed to meet urgent political deadlines will retain essential features of authentic curriculum-driven assessments. Part I presents research evidence on…