This article considers some patterns of ranking higher education institutions which are used in the Russian Federation to reveal strengths and weaknesses in meeting the national individual, societal and state-related needs, as well as those of the international academic community concerning relevant information on Russian higher education…

Two years after the first low-cost radio-frequency audience response system using clickers was introduced for college classrooms, at least six different systems are on the market. Their features and user-friendliness are evolving rapidly, driven by competition and improving technology. The proliferation of different systems is putting pressure on…

Quality assurance in medical education is a complex matter. Institutions and external critics of accreditation processes often question the objectivity of peer judgements. Members of accreditation panels similarly often feel at a loss for a lack of mechanisms to help them arrive at a collective assessment of the performance of an institution. In…

Introduction: Heightened concerns about industry influence on continuing medical education (CME) have prompted tighter controls on the management of commercial funding and conflict of interest. As a result, CME providers must closely monitor their activities and intervene if bias or noncompliance with accreditation standards is likely. Potential…

Background/Context: The nationwide preoccupation with accountability continues to grow, with teacher credentialing programs facing growing scrutiny through state-mandated accountability systems. In response to Senate Bill 2042 passed by the California state legislature in 1998, the California Commission for Teacher Credentialing (CCTC) established…

Education in the United States is becoming increasingly standardized, with the standards being initiated at the national level and then trickling down to the state level and finally the local level. Yet, this top-down approach to educational standards carries with it significant limitations, such as loss of local autonomy and restrictions on the…

The "Evaluation of the National Assessment of Educational Progress: Study Reports" describes the special studies that comprised the design of the evaluation. In the Final Report, the authors presented a practical discussion of the evaluation studies to its primary, intended audience, namely policymakers. On this accompanying CD, readers…

This study investigated the intrajudge consistency of Angoff-based item performance estimates. The examination used was a certification examination in an emergency medicine specialty. Ten expert panelists rated the same 24 items twice during an operational standard setting study. Results indicate that the panelists were highly consistent, in terms…

Elements of arbitrariness in the standard setting process are explored, and an alternative to the use of cut scores is presented. The first part of the paper analyzes the use of cut scores in large-scale assessments, discussing three different functions: (1) cut scores define the qualifications used in assessments; (2) they simplify the reporting…

Discussed are nine questions regarding standard setting issues in educational testing: (1) Should normative or content-referenced standards be used? (2) Different standard setting methods yield different results. Does this finding present a problem? (3) Assess the adequacy of the grounding of various methods of standard setting in psychological…

Three studies assessed the psychometric characteristics of the Direct Standard Setting Method (DSSM). The Angoff technique was also used in each study. The DSSM requires judges to consider an examination 10 items at a time and determine the minimum items in that set a candidate should answer correctly to receive the credential. Nine judges set a…

Cut scores obtained for licensure tests using different judgmental methods of standard setting (holistic, test blueprint, Angoff, and modified Angoff) were compared. Nineteen educators and practitioners participated in this study as judges. Pre- and post-test feedback (feedback of total- and low-group item p-value) ratings were obtained under the…

This paper is a report of a study designed to develop recommendations on minimum qualifying scores for National Teacher Examinations (NTE) that are valid for certification and endorsement in Tennessee. The functions performed in the review of the NTE Core Battery and Specialty Area tests were conceptualized as panel activities. The number of…

Discrepancies among the standards produced by different criterion-referenced standard-setting techniques may be the result of a failure to adequately define the minimally competent candidate. Current research in this area is reviewed in terms of three categories: studies in which no formal assistance in conceptualization is given to judges,…

This paper evaluates the magnitude of the total error in estimates of the difference between an examinee's domain score and the cutoff score. An observed score based on a random sample of items from the domain, and an estimated cutoff score derived from a judgmental standard setting procedure are assumed. The work of Brennan and Lockwood (1980) is…