This paper contains a comparison of descriptive information based on analyses of pilot and live administrations of the Alabama High School Graduation Examination (AHSGE). The test is composed of three subject tests: Reading, Mathematics, and Language. The study was intended to validate the test development procedure by comparing difficulty levels of common test items measuring competencies on the initial item pilot and on the live administrations. Data were collected during two phases of pilot testing and four live administrations. Public schools in Alabama with at least 50 students were randomly sampled to participate in the pilot procedure. Data were analyzed, and difficulty levels and discrimination indexes for all pilot items were calculated, based on the proportion of students completing the item correctly. A second pilot test involved testing of forms. Following live administrations, common items were compared with the initial item pilot. The comparison demonstrated the stability of the difficulty level and supported the validity of the test development procedure. Although a higher percentage of students passed items on the live administration, the slight increase in percentage seemed to be consistent from one form to another. (LMO)