This study explores the use of the modified caution index (MCI) for identifying judges whose patterns of recommendations suggest that their judgments might be based on incomplete information, flawed reasoning, or inattention to their standard-setting tasks. It also examines the effect on test standards and passing rates when the test standards of judges whose recommendations appear aberrant are eliminated. Data for this study were collected in a project to establish standards on the North Carolina High School Competency Tests. In that study, judges provided recommendations on an appropriate standard for a 120-item reading test (n=83) and for a 120-item math test (n=84). Matrices of item-by-item recommendations provided by judges were ordered by total recommended test standard, and by number of item endorsements. Once MCI had been computed for each judge, distributions of MCI were constructed for each subtest, and the effects of eliminating judges with MCI greater than selected threshold values were determined. Results indicate that screening judges on the basis of their MCI values can make a substantial difference in the resulting test standard and in the resulting failure rate on a high school competency test. (PN)