A summary is provided of the rationale for questioning the applicability of classical reliability measures to criterion referenced tests; an extension of the classical theory of true and error scores to incorporate a theory of dichotomous decisions; a presentation of the mean split-half coefficient of agreement, a single-administration test index designed to measure the internal consistency of dichotomous classifications; and information concerning the properties, under varying conditions, of this new coefficient and several other single-administration test indices, as well as their interrelationships. Simulated data were used to provide answers to questions about the behavior of coefficient beta relative to variations in score distribution, criterion level, number of examinees, number of items, and certain basic test statistics. It was determined that coefficient beta increases as the number of items increases, but in a manner different from that predicted by the Spearman-Brown prophecy formula. It was also shown that the value of the coefficient increases as the bulk of scores departs from the criterion cutoff. Relationships between coefficient beta and other test indices are presented. Most prominent among these is the indication that for unimodal score distributions, coefficient beta and Livingston's criterion referenced reliability coefficient have similar ranges of value and fluctuations over criterion level, whereas this relationship does not hold for bimodal distributions, since coefficient beta is sensitive to the mode(s) of the score distribution while Livingston's coefficient is sensitive to the test mean. (Author/RC)