This study aims to demonstrate the optimal way to determine the cut-off score to be used to interpret the total scores obtained from an achievement test or scale using the Artificial Neural Networks method. To this end, the multiple-choice item responses in the Booklet-11 Mathematics subtest at the 8th grade level in the TIMSS 2015 Turkey sample…

In this article, we consolidate and present in one place what is known about quality indicators for setting standards so that stakeholders may be able to recognize the signs of standard-setting quality. We use the context of setting standards to associate English language test scores with language proficiency descriptions such as those presented…

Critical thinking is an essential skill for learners and teachers alike. Therefore, it is essential that educators be given practical strategies for improving their critical thinking skills as well as methods to effectively provide critical thinking skills to their students. The "Handbook of Research on Critical Thinking and Teacher Education…

This study compares the efficacy of different strategies for translating item-level, proportion-correct standard-setting judgments into a theta-metric test cutoff score for use with item response theory (IRT) scoring, using Monte Carlo methods. Simulated Angoff-type ratings, consisting of 1,000 independent 75 Item x13 Rater matrices, were…

Purpose: The quality of an education system or a comparative international assessment refers more and more to quantitative parameters, i.e. "educational indicators." The paper aims to analyse the structure of several educational indicators and indicator systems and answer the question "What can educational indicators achieve?"…

While standard setting procedures are typically discussed in terms of deriving a reasonable cutting score for a given form of a test, the situation may be structured such that the standard has been mandated without regard to the test form itself. This situation may result either through legislative or policy actions and may be a fait accompli by…

In recent years educators have been utilizing judgmental methods, such as the ones advocated by Ebel and Angoff, to set minimum competency standards on test items. This study was designed to investigate the reliability and validity of these two procedures in setting minimum levels of performance on 175 vocational evaluator competency statements.…

Setting standards for the purpose of certification is frequently performed using judgmental techniques such as the Angoff method. This study was performed to identify types of items that judges find hard to rate accurately, that is, types of items on which examinees perform differently than predicted by the judges. Once identified these item types…

A systematic method for compromise between absolute and relative examination standards is proposed. The passing score is assumed to be related to expected pass rate through a simple linear function. Results define a function relating the percentage of successful candidates given a specified passing score to the passing score. (Author/DWH)

With the exception of the procedures developed by Porter and colleagues (Porter, 2002), other methods of defining and measuring alignment are essentially limited to alignment between tests and standards. Porter's procedures have been generalized to investigating the alignment between content standards, tests, textbooks, and even classroom…

Traditionally, judgmental standard setting methods have been used exclusively at the test item level. In this study, the Ebel and Angoff methods of standard setting were utilized to determine minimum competency standards on a list of 175 identified competency statements for vocational evaluators. The following research questions were addressed:…

Exams which are periodically updated and revised also require a revision of the cut score to ensure that examinees who take the original form of the test, and those taking the revised form, are scored on an equivalent basis. This paper discusses available standard setting procedures as applied to establishing minimum passing scores for licensing…

This paper is a report of a study designed to develop recommendations on minimum qualifying scores for National Teacher Examinations (NTE) that are valid for certification and endorsement in Tennessee. The functions performed in the review of the NTE Core Battery and Specialty Area tests were conceptualized as panel activities. The number of…

This report investigates an area of uncertainty in using the Angoff method for setting standards, namely whether or not a judge's conceptualizations of borderline group performance are realistic. Ratings are usually made with reference to the performance of this hypothetical group, therefore the Angoff method's success is dependent on this point.…

The officially reported impressive rises in standards in mathematics and English in primary schools since 1995 are challenged. The article looks at the increases from four different perspectives. First, the general pattern of change is considered. Secondly, the statutory test data are compared with the results from several different studies. These…