This study examined and refined a mathematics skills instrument used to predict success in an introductory statistics course and to measure improvement in mathematics skills. Alternate forms of a 28-item true-false test were administered as a pretest and posttest to 18 undergraduate and 19 graduate students enrolled in introductory educational statistics courses at the University of Alabama. Cronbach's alpha and corrected item-total correlations were computed for the pretest. Item analysis resulted in the elimination of 10 items from the original instrument. The alpha coefficient for the resulting 18-item instrument was .75. Total pretest scores were correlated with each of three course exams and with course average. Results revealed significant correlations with examinations administered after the first examination for the sample as a whole. However, correlations for the two separate groups revealed significant results only for undergraduates. A groups-by-tests repeated measures design was employed to detect between-group differences and mathematics skills improvement from pretest to posttest. Both groups improved, with greater improvement shown for undergraduates, but no conclusions could be drawn regarding the reason for the improvement. (Author/JGL)