Formulas are presented for decision reliability and for classification validity for mastery/nonmastery decisions based on criterion referenced tests. Two item parameters are used: the probability of a master answering an item correctly, and the probability of a nonmaster answering an item incorrectly. The theory explores the relationships of reliability and validity to test length, cutting scores, and the two item parameters. The model handles the cases in which the item pool either contains items with equal or with unequal item parameters. A guide is presented to enable practitioners to generate reliability and validity matrices for heterogeneous item pools. Bayesian methods are explored. (Author/CTM)