A framework for integrating the many conceptions of criterion-referenced testing is provided, which illustrates that there is no single type of test that can be identified as the prototype criterion-referenced test. The use of one kind of criterion-referenced test may not be applicable to another. (Author/RL)

Discusses problems in using traditional measures of reliability for criterion-referenced tests (CRTs) and describes two approaches to reliability for CRTs: estimates sensitive to all measures of error; and estimates of consistency in test outcome. Compares the two approaches and proposes recommendations for interpretation and use. (Author/LRW)

A relatively simple method is developed to obtain confidence intervals for a student's proportion of domain mastery in criterion-referenced or mastery measurement situations. The method uses the binomial distribution as a model for the student's scores under hypothetically repeated assessments, and it makes use of widely available "F"…

Two types of test scores are often used for determining eligibility for early intervention services: standard scores and developmental-age scores. Because they are derived using different methods, these two types of scores do not provide equivalent information. Characteristics of norm-referenced and criterion-referenced tests and the scores…

In educational testing, subscores may be provided based on a portion of the items from a larger test. One consideration in evaluation of such subscores is their ability to predict a criterion score. Two limitations on prediction exist. The first, which is well known, is that the coefficient of determination for linear prediction of the criterion…

In 2005, the undergraduate advisory committee at the University of Toronto Mississauga found that across all disciplines, writing proficiency was the skill weakness that generated the greatest concern. Students reported that they often found writing tasks intimidating, and suggested that effective feedback and guidance would improve their writing.…

This article attempts to present a novel application of a method of measuring accuracy for academic success predictors that could be used as a standard. This procedure is known as the receiver operating characteristic (ROC) curve, which comes from statistical decision techniques. The statistical prediction techniques provide predictor models and…

Research Findings: We examined the reliability and validity of the language, literacy, and mathematics domains of "Work Sampling for Head Start" (WSHS), an observational assessment designed for 3- and 4-year-olds. Participants included 112 children who were enrolled over a two-year period in Head Start and a number of other programs…

The study investigated the feasibility of adapting the coefficient k introduced by Cohen (1960) and elaborated by Swaminathan, Hambleton, and Algina (1974) to an internal consistency estimate for criterion referenced tests in single test administrations. The authors proposed the use of k as an internal consistency estimate by logically dividing…

Single-administration procedures for estimating the coefficient of agreement, a reliability index for criterion referenced tests, were recently developed by Subkoviak. The procedures require a distributional assumption for errors of measurement and an estimate of each examinee's true score. A computer simulation of tests composed of items that…

This study examined the effectiveness of two different domain definition strategies in achieving homogeneity of criterion-referenced test items. The argument was tested regarding the extent to which item writers following the Instructional Objectives Exchange (IOX) domain definition strategy for a cognitive skill generate items that are more…

A framework, or classification scheme, is provided for displaying the spectrum of criterion-referenced tests. This framework illustrates that no single type of test can be identified as the absolute prototype criterion-referenced test. It is shown that over the past 115 years criterion-referenced testing has grown to be a many-faceted concept, its…

Teachers should be evaluated chiefly by the results they produce. Those results will not be properly detected through the use of norm-referenced achievement tests (NRT) due to the following major deficits of NRT: (1) the descriptions of what is measured by NRT are far too loose; (2) evaluative expectations are unclear; (3) the necessity for NRT to…

This annotated bibliography lists currently available criterion-referenced measures. It includes measures appropriate for a wide range of goals and across a variety of subjects. The inclusion of a test in this bibliography should not be construed as a recommendation of that test. (Author/DEP)