This paper offers answers to nine important questions concerning standard-setting issues and methods: (1) Should normative or content-referenced standards be used? (2) Different standard-setting methods yield different results. What is your reaction to this finding? That of your clients? Does this finding present a problem for the application of…

This paper presents the author's responses to nine questions addressed to the standard-setting symposium participants. A central theme in this assessment is that one cannot clearly demonstrate, a priori, that either a normative or a content-referenced approach is superior. Most methods used to date have been insufficiently grounded in…

A new approach based on Rasch measurement theory is described for examining the quality of ratings from standard-setting judges. Ratings of nine judges for 213 items on a nursing examination show that judges vary in their views of the essential items for nursing certification, with statistically significant variability in the judged essentiality…

A conceptual framework is proposed for a psychometric theory of standard setting. The framework suggests that participants in a standard setting process (panelists) develop an internal, intended standard as a result of training and the participant's background. The goal of a standard setting process is to convert panelists' intended standards to…

We look at some of the theoretical and methodological issues underlying international university ranking systems and, in particular, their conceptual connection with the idea of excellence. We then turn to a critical examination of the two best-known international university ranking systems--the "Times Higher Education Supplement (THES)" World…

Standard setting is a fairly widespread activity in educational and psychological measurement, but there is no formal psychometric theory to guide the development of standard setting methodology. This paper presents a conceptual framework for such a psychometric theory and uses the conceptual framework to analyze a number of methods for setting…

Traditionally, judgmental standard setting methods have been used exclusively at the test item level. In this study, the Ebel and Angoff methods of standard setting were utilized to determine minimum competency standards on a list of 175 identified competency statements for vocational evaluators. The following research questions were addressed:…

The Achievement-Levels Setting (ALS) process for the National Assessment of Educational Progress (NAEP) resulted in numerical cutscores on the NAEP score scale representing the performance standards for three achievement levels: Basic, Proficient, and Advanced. This paper focuses on an important, but less researched, aspect of the standard setting…

A brief summary of standard setting knowledge is presented, derived from about 20 methods that utilize a judgmental review process, the approach most relevant to the standard-setting strategies proposed in this special issue. Criteria for judging effectiveness and critiques of the methods discussed in the issue are offered. (SLD)

Discusses the roles played by participants and the processes involved in a provincial standards-setting exercise for a large-scale assessment of student skills. Offers perspectives of a policy maker outside the exercise and an insider participant. Describes alternate notions of representativeness, stakes, and significance. (SLD)

The Bookmark standard-setting procedure was developed to address the perceived problems with the most popular method for setting cut-scores: the Angoff procedure (Angoff, 1971). The purposes of this article are to review the Bookmark procedure and evaluate it in terms of Berk's (1986) criteria for evaluating cut-score setting methods. The…

The increase in litigation by students who are dissatisfied with their assessment, and to a lesser extent the time and monetary costs of student appeals makes it imperative that institutions adopt a robust assessment strategy. The concern of consumers with respect to professional services offered by students after graduation is also an issue. This…

The burgeoning field of Out-of-School Time provides academic and social services to K-12 students during the time that they are not in school. Hundreds of service providers offer a range of programs that target various student populations. Many tools exist to measure program performance, but no uniform evaluation methods exist to inform funders…

This article considers some patterns of ranking higher education institutions which are used in the Russian Federation to reveal strengths and weaknesses in meeting the national individual, societal and state-related needs, as well as those of the international academic community concerning relevant information on Russian higher education…

Introduction: Heightened concerns about industry influence on continuing medical education (CME) have prompted tighter controls on the management of commercial funding and conflict of interest. As a result, CME providers must closely monitor their activities and intervene if bias or noncompliance with accreditation standards is likely. Potential…