The comparability of examinations in different subjects has been a controversial topic for many years and a number of criticisms have been made of statistical approaches to estimating the "difficulties" of achieving particular grades in different subjects. This paper argues that if comparability is understood in terms of a linking…

The No Child Left Behind Act and the accountability movement in public education caused many states to develop criterion-referenced academic achievement tests. Scores from these tests are often used to make high stakes decisions. Even so, these tests typically do not receive independent psychometric scrutiny. We evaluated the 2005 Arkansas…

It is proposed that the reliability of criterion-referenced test scores be defined in terms of the consistency of the decision-making process across repeated administrations of the test. (Author/RC)

The kappamax reliability index of domain-referenced tests is defined as the upper bound of kappa when all possibile cutoff scores are considered. Computational procedures for kappamax are described, as well as its approximation for long tests, based on Kuder-Richardson formula 21. The sampling error of kappamax, and the effects of test length and…

Predictive Sample Reuse is an alternative to parameter estimation for solving a variety of statistical problems. The technique is described and its relevance to criterion referenced measurement discussed. An item analysis procedure which does not require multiple test administrations or variability in test or item scores and which adheres to the…

Within the last decade, a wealth of literature has appeared debating the merits, use, construction, and interpretation of criterion referenced tests. This is a bibliography of articles on a wide range of topics on criterion referenced testing subjects. Also included are articles on mastery and domain referenced testing. The articles cited were…

A strict equivalence presupposed by Kaiser and Michael to derive the coefficient of "domain validity" is defensible only as a biased approximation. But then, it is far from clear what psychometric significance this coefficient has in the first place. (Author)

Professional educators as goal formulators are presented as the responsible agents of accountability. Norm- and criterion-referenced testing are compared and suggestions for accountability and goal development are given. (RAO)

An index for the dependability of mastery tests is described. Assumptions necessary for the index and the mathematical development of the index are provided. (Author/JKS)

This digest introduces ways of responding to the call for criterion-referenced information using Bayes' Theorem, a method that was coupled with criterion-referenced testing in the early 1970s (see R. Hambleton and M. Novick, 1973). To illustrate Bayes' Theorem, an example is given in which the goal is to classify an examinee as being a master or…

Discusses advantages and disadvantages of norm-referenced and criterion-referenced tests. (JF)

A discussion of the need to utilize a training approach based on job performance requirements and criterion tests." (Author/AK)

Two indices for assessing the efficiency of decisions in mastery testing are proposed. The indices are generalizations of the raw agreement index and the kappa index. Empirical examples of these indices are given. (Author/JKS)

Techniques for classifying individuals into mastery or non-mastery status based on criterion-referenced test performance are investigated. Extensions of mastery classification techniques developed by Macready and Dayton are presented. The techniques are based on latent class and quasi-independence models. (Author/JKS)