This classroom experiment investigates the effects of adding representational pictures to multiple-choice and constructed-response test items to understand the role of the response format for the multimedia effect in testing. Participants were 575 fifth- and sixth-graders who answered 28 science test items--seven items in each of four experimental…

This comparative research study examined Grade Five Malaysian Aboriginal pupils' mathematical performance on 30 computation and 20 word problem items in the academic Malay language and community "Temiar" language. The items were constructed in the Malay language before adapting and audio recording into the "Temiar" language…

Language is frequently discussed as barrier to mathematics word problems. Hence this paper presents the initial findings of a linguistic analysis of numeracy skills test sample items. The theoretical perspective of multi-modal text analysis underpinned this study, in which data was extracted from the ten sample numeracy test items released by the…

In this study, we developed a three-dimensional framework to characterize post-secondary Calculus I final exams. Our "Exam Characterization Framework" (ECF) classifies individual exam items according to the cognitive demand required to answer the item, the representation of both the task statement and the solution, and the item's format.…

In this article we explore the use of pictorial glossaries as an accommodation for English learners (ELs) with entry and emerging levels of English language proficiency. Drawing on survey responses from 98 middle school ELs and cognitive interviews with 10 of the survey participants, we examined the participants' preferences and experiences with…

This report provides a description of the development process, field testing, and psychometric properties of a student mathematics test designed to assess grades K, 1, and 2 student abilities. The test was administered to 4,486 participating grade K, 1, and 2 students in 67 schools located in 10 public school districts in Florida during fall 2015.…

This study aims at determining to what extent visuals contribute to success in non-routine problem solving process and what types of errors are made when solving with visuals. Comparative model was utilized for identifying the effect of visuals on student achievement, and clinical interview technique was used to determine the types of errors. In…

This study investigates whether word problems and mathematically expressed items can be used interchangeably regardless of their linguistic complexities. A sample of sixth grade students was given two forms of a mathematics assessment. The first form included mathematics items with mathematical terms, expressions, and equations whereas the second…

This study investigated whether the linguistic complexity of items leads to gender differential item functioning (DIF) on mathematics assessments. Two forms of a mathematics test were developed. The first form consisted of algebra items based on mathematical expressions, terms, and equations. In the second form, the same items were written as word…

The research described in this report is a randomized controlled trial in which seventh- and eighth-grade students were randomly assigned to complete a set of 25 math questions delivered with either standard language or language that had undergone "linguistic modification" by the research team. The purpose of the study was to assess the…

Cognitive complexity level is important for measuring both aptitude and achievement in large-scale testing. Tests for standards-based assessment of mathematics, for example, often include cognitive complexity level in the test blueprint. However, little research exists on how mathematics items can be designed to vary in cognitive complexity level.…

In this article, Maria Martiniello reports the findings of a study of the linguistic complexity of math word problems that were found to exhibit differential item functioning for English-language learners (ELLs) and non-ELLs taking the Massachusetts Comprehensive Assessment System (MCAS) fourth-grade math test. It builds on prior research showing…

This article examines nonmathematical linguistic complexity as a source of differential item functioning (DIF) in math word problems for English language learners (ELLs). Specifically, this study investigates the relationship between item measures of linguistic complexity, nonlinguistic forms of representation and DIF measures based on item…

This study focuses on the relationship between item modeling and evidence-centered design (ECD); it considers how an appropriately generalized item modeling software tool can support systematic identification and exploitation of task-model variables, and then examines the feasibility of this goal, using linear-equation items as a test case. The…

Simple arithmetic word problems are often featured in elementary school education. One type of problem, "compare with unknown reference set," ranks among the most difficult to solve. Differences in item difficulty for compare problems with unknown reference set are observed depending on the direction of the relational statement (more than vs. less…