Personnel standards are the foundations for how states and nations approve a program, engage in systemic assessment, and provide effective professional development to its early childhood professionals. However, despite the extensive use of paraprofessionals in early intervention/early childhood special education programs, there is a lack of…

Standard-setting studies utilizing procedures such as the Bookmark or Angoff methods are just one component of the complete standard-setting process. Decision makers ultimately must determine what they believe to be the most appropriate standard or cut score to use, employing the input of the standard-setting panelists as one piece of information…

The Many-Facet Rasch (MFR) Model is traditionally used to evaluate the quality of ratings on constructed response assessments; however, it can also be used to evaluate the quality of judgments from panel-based standard setting procedures. The current study illustrates the use of the MFR Model by examining the quality of ratings obtained from a…

The Arizona English Language Learners Assessment (AZELLA) is used by the Arizona Department of Education to determine which children should receive English support services. AZELLA results are used to determine if children are either proficient in English or have English language skills in one of four pre-proficient categories (pre-emergent,…

Judgmental standard setting approaches rely on the perceptions of experts about examinee performance on a test. Traditional standard setting methods ask panelists to estimate how well a randomly selected hypothetical examinee who is representative of a well-defined target group, usually a minimally competent candidate (MCC), will perform on each…

In recent years educators have been utilizing judgmental methods, such as the ones advocated by Ebel and Angoff, to set minimum competency standards on test items. This study was designed to investigate the reliability and validity of these two procedures in setting minimum levels of performance on 175 vocational evaluator competency statements.…

Judgmental standard setting methods, such as the Angoff method (W. Angoff, 1971), use item performance estimates as the basis for determining the minimum passing score (MPS). Therefore the accuracy of these item performance estimates is crucial to the validity of the resulting MPS. Recent researchers (L. Shepard, 1994; J. Impara, 1997) have called…

Five forms of a basic science examination administered over three years in a national board testing program were analyzed to determine the stability of judged cut scores. Results indicate that cut scores derived from the modified Nedelsky procedure were within equating error of the Rasch equated cut scores over the five administrations. (SLD)

Outlines a framework for considering the validity of standards-based score interpretations and then considers the potential roles of different stakeholder groups and other participants in that process. Suggests study of a new standard-setting method, the "briefing book," which would describe alternative cut scores. (SLD)

Judgmental standard setting approaches rely on the perceptions of experts about examinee performance on a test. Traditional standard setting methods ask panelists to predict the probability that a randomly selected, hypothetical minimally competent candidate (MCC) will correctly answer test questions. Item performance predictions are difficult for…

Results of the fall 1993 administration of part two of the Medical Council of Canada's Evaluating Examination for 744 candidates provided evidence of the consistency of the pass/fail and cutting score definitions for the objective- structured clinical examination stations used across examiners. These results support the validity of this…

The National Assessment Governing Board and the National Center for Education Statistics sponsored a Joint Conference on Standard Setting for Large-Scale Assessments to provide a forum for technical and policy issues relevant to setting standards at local, state, and national levels. This executive summary conveys the essence of the conference by…

A framework is provided for examining the validity of performance standards for high-stakes achievement tests. Assumptions that need to be evaluated are that the passing score corresponds to specified performance standards and that the specified standard is reasonable given the purpose of the decision. (SLD)

The "Standards for Educational and Psychological Testing" (1985) and the "Uniform Guidelines on Employee Selection Procedures" (1978) provide no clear guidance on choosing methods for setting standards on teacher certification tests, although they require that practices be well-documented and produce reasonable results, and…

The National Assessment Governing Board and the National Center for Education Statistics sponsored a Joint Conference on Standard Setting for Large-Scale Assessments to provide a forum for technical and policy issues relevant to setting standards at local, state, and national levels. Volume I contains an executive summary of the conference and…