Recent developments in the field of artificial intelligence allow for improved performance in the automated assessment of extended response items in mathematics, potentially allowing for the scoring of these items cheaply and at scale. This study details the grand prize-winning approach to developing large language models (LLMs) to automatically…

This study aimed to examine an assumption regarding whether generative artificial intelligence (GAI) tools can overcome the cognitive intensity that humans suffer when solving problems. We examine the performance of ChatGPT and GPT-4 on NAEP science assessments and compare their performance to students by cognitive demands of the items. Fifty-four…

This exploratory study investigated the behaviors and content of onscreen calculator usage by a nationally representative sample of eighth-grade students who responded to items from the 2017 National Assessment of Educational Progress mathematics assessment. Meaningful features were generated from the process data to infer whether students…

This investigation explores the relationship between the use of digital pencil and mathematical problem-solving accuracy among 1,530 students with learning disabilities (LD) and 25,400 general education (GE) peers from the 2017 digital National Assessment of Educational Progress mathematics assessment. The term "digital pencil" in this…

This investigation explores the relationship between the use of digital pencil and mathematical problem-solving accuracy among 1,530 students with learning disabilities (LD) and 25,400 general education (GE) peers from the 2017 digital National Assessment of Educational Progress mathematics assessment. The term "digital pencil" in this…

In this study, we explore the application of process mining techniques on assessment log data to explore problem-solving strategies in Algebra. By analyzing sequences of student activities, we demonstrate the significant potential of process mining in identifying problem-solving strategies that lead to successful and unsuccessful outcomes. Our…

For autistic students receiving special education services, little is known about their relative strengths, weaknesses, and enjoyment across different math content areas; their overall math interest and persistence are also not well-studied. Using the 2017 eighth-grade National Assessment of Education Progress data, this study finds, relative to…

According to the National Assessment of Educational Progress (NAEP), US reading and math scores have recently dropped. This decline is believed to be the result of the school closings that occurred during the COVID-19 pandemic. The drop likely contributed to the increase in pressure educators are feeling to teach in a way that leads to higher test…

In 2017, the mathematics assessments that are part of the National Assessment of Educational Progress (NAEP) program underwent a transformation shifting the administration from paper-and-pencil formats to digitally-based assessments (DBA). This shift introduced new interactive item types that bring rich process data and tremendous opportunities to…

Many large-scale educational surveys have moved from linear form design to multistage testing (MST) design. One advantage of MST is that it can provide more accurate latent trait [theta] estimates using fewer items than required by linear tests. However, MST generates incomplete response data by design; hence, questions remain as to how to…

The present study examines the features and quality of visual representations (VRs) created by middle school students with learning disabilities and difficulties in mathematics in response to a released fraction item from the National Assessment of Educational Progress (NAEP). Relations between VR quality and scores on other measures of fraction…

The link between reading and mathematics achievement is well known, and an important question is whether readability factors in mathematics problems are differentially impacting student groups. Using 20 years of data from the National Assessment of Educational Progress and the Trends in International Mathematics and Science Study, we examine how…

Today's world is shaped by technology and products engineered to serve society's needs and wants. The knowledge and skills required to develop and use these products are changing at an exponential rate (NASE&M, 2017; NRC, 2011). In order to understand and meet future challenges, it is vital that the United States produce technology- and…

The Alabama State Department of Education (ALSDE) was grappling with several issues including: (1) A respectable rate of growth in student achievement for both fourth- and eighth-grade math in the state of Alabama, as measured by the National Assessment of Educational Progress (NAEP), but below the national average; and (2) A growing concern about…

In this article, we use multimodality to examine how bilingual students interact with an area task from the National Assessment of Educational Progress in task-based interviews. Using vignettes, we demonstrate how some of these students manipulate the concrete materials, and use gestures, as a primary form of structuring their explanations and…