This study compared the performance of English language learners and native speakers of English on mathematics word problems from the National Assessment of Educational Progress (NAEP) tests and investigated whether modifying the linguistic structures in the test items affected student test performance. The study began with the analysis of existing NAEP data. These analyses strongly suggested that students' language background impacts their performance. Two separate field studies were then conducted. For the first, a study of student perceptions, 36 eighth graders from the greater Los Angeles area were interviewed. These students were administered the original mathematics items and parallel revised items (with simplified language) in a structured interview format to investigate their perceptions and preferences. In the second field study, the Accuracy Test Study, 1,174 eighth graders took paper-and-pencil mathematics tests with 10 original NAEP items, 10 items with linguistic modifications, and 5 noncomplex control items. Overall, results show that students in the English as a Second Language categories, especially those in the lower levels, had considerably lower mathematics performance than other students. Revising the items to make them less linguistically complex helped some students, although such improvement was not statistically significant. (Contains 44 references.) (SLD)