Multiple matrix sampling is applied to a variation of Angoff's standard setting method. Thirty-six experts (internists) and 190 items were divided into five groups, and borderline examinee performance was estimated. There was some variability in the cutting scores produced by the individual groups, but various components were well estimated. (SLD)

An ignored aspect of standard setting, namely the possibility that Angoff or Nedelsky judges specify inconsistent probabilities (e.g., low probabilities for easy items but large probabilities for hard items) is explored. A latent trait method is proposed to estimate such misspecifications, and an index of consistency is defined. (Author/PN)