Focusing on the use of the Metropolitan Achievement Test (MAT) with Project Follow Through evaluations, this report represents an extended discussion of the appropriateness of standardized tests for evaluation and policy research. First, the development of the MAT and its roots in the Army Alpha intelligence test are described in detail. The next section discusses metric problems--that is, the types of scores standardized tests yield. The report recommends the use of the publisher's standard score, citing reasons why raw scores, percentiles, and grade equivalents are subject to misinterpretation. Psychometric problems--those concerned with test validity--are also discussed. It is argued that the use of standardized tests is not now justified because the tests may measure test wiseness, achievement motivation, general ability, and white middle class values rather than specific skills that schools are trying to teach. In the final section, recommendations are made for designing and validating new tests to fit particular goals: assessment and evaluation, identification of effective programs, diagnosis and prescriptions, and educational needs assessment. The appendix contains a history of the Army Alpha, notes on the norm referenced/criterion referenced distinction in testing, and results of the MAT analyses regarding the Philadelphia Follow Through Planned Variation evaluation. (CP)