This paper describes a study that investigated the interchangeability of four different assessment methods for measuring middle-school students' understanding of science concepts. The four methods compared were hands-on tasks with associated multiple-choice and written justification items, written analogues of the hands-on tasks, and two types of multiple-choice items that were not related to hands-on or written analogue tasks. Participants were 662 seventh and eighth graders from 21 classes in 5 Los Angeles County (California) schools. Some students took the hands-on test before the written test, and some students took the written test before the hands-on test. Observed and disattenuated correlations were examined. Multivariate generalizability analysis was used to obtain disattenuated correlations. The results indicate that hands-on and written analogue tests are not interchangeable, but multiple-choice choice and written justification items linked to hands-on and written analogues could be considered interchangeable if correlations between 0.76 and 0.96 are an acceptable criterion for interchangeability. In addition, a number of interesting order effects were found. (Contains 1 figure, 12 tables, and 18 references.) (Author/SLD)