Achieve, through support from the Noyce Foundation, examined ten sets of international standards with the intent of informing the development of both the conceptual framework and Next Generation Science Standards. Achieve selected countries based on their strong performance on international assessments and/or their economic, political, or cultural…

This study compares the efficacy of different strategies for translating item-level, proportion-correct standard-setting judgments into a theta-metric test cutoff score for use with item response theory (IRT) scoring, using Monte Carlo methods. Simulated Angoff-type ratings, consisting of 1,000 independent 75 Item x13 Rater matrices, were…

The Angoff method is a widely used procedure for setting pass scores in vocational examinations, in which the awarders estimate the performance of minimally competent candidates (MCCs) on each test item. Within the context of some UK vocational examinations, the procedure consists of two stages: after making the first round of estimates, awarders…

In response to concerns over the need for a scientifically literate workforce, increasing the STEM pipeline, and aging science standards documents, the scientific and science education communities are embarking on the development of a new conceptual framework for science, led by the National Research Council (NRC), and aligned next generation…

University rankings are having a profound effect on both higher education systems and individual universities. In this paper we outline these effects, discuss the desirable characteristics of a good ranking methodology and document existing practice, with an emphasis on the two main international rankings (Shanghai Jiao Tong and THES-QS). We take…

"Classification" and "ranking" are tools for comparing colleges and universities, albeit for different purposes. Although "classification" focuses on identifying similarities rather than ordering institutions, it is closely intertwined with "ranking", whether directly through its content, structure, and…

Recently, there has been a significant expansion in the use of performance assessment in large scale testing programs. Although there has been significant support from curriculum and policy stakeholders, the technical feasibility of large scale performance assessments has remained a question. This report is intended to contribute to the debate by…

Forensic versus traditional accounting and auditing are compared and contrasted. Evidence gathering is detailed. Forensic science and fraud symptoms are explained. Criminalists, expert testimony and corporate governance are presented.

In this study, a variation of the bookmark standard setting procedure for passage-based tests is proposed in which separate ordered item booklets are created for the items associated with each passage. This variation is compared to the traditional bookmark procedure for a fifth-grade reading test. The results showed that the single-passage…

Purpose: This paper aims to explain the distinction between educational standards and learning outcomes and to indicate the problems that potentially arise when a learning outcomes approach is applied to a qualification meta-framework like the European Qualification Framework, or indeed to national qualification frameworks.…

Research indicating that different cut-off points result from the use of different standard-setting techniques leaves decision makers with a disturbing dilemma: Which standard-setting method is best? This investigation of the reliability and validity of 10 different standard-setting approaches was designed to provide information that might help…

This report focuses on two aspects of teacher supply and quality: (a) the under-representation of racial/ethnic minorities, especially African Americans in the teaching pool, and (b) teacher candidates' performance on licensure assessments, including general skills tests in reading, writing, and mathematics (known as Praxis I[R]) and selected…

Using meta-analysis, this research takes a look at studies included in a meta-analysis by R. Jaeger (1989) that compared the cut score set by one standard setting method with that set by another. This meta-analysis looks beyond Jaeger's studies to select 10 from the research literature. Each compared at least two types of standard setting method.…

In recent years educators have been utilizing judgmental methods, such as the ones advocated by Ebel and Angoff, to set minimum competency standards on test items. This study was designed to investigate the reliability and validity of these two procedures in setting minimum levels of performance on 175 vocational evaluator competency statements.…

The use of item response theory models for analyzing the results of judgmental standard setting studies (the Angoff technique) for establishing minimum pass levels is discussed. A comparison of three methods indicates the traditional approach may not be best. A procedure based on generalizability theory is suggested. (GDC)