The standard methodology for many large-scale assessments in education involves regressing latent variables on numerous contextual variables to estimate proficiency distributions. To reduce the number of contextual variables used in the regression and improve estimation, we propose and evaluate principal component analysis on the covariance matrix…

This study examines the impact of early Algebra I coursework on advanced Carnegie credits among 12th graders from diverse racial/ethnic backgrounds, using data from the NCES HSTS (1990-2019). Findings indicate that early Algebra students, particularly Black and Latinx, earn more advanced credits, revealing a widening gap in advanced course…

The "2nd Annual WPI-UMASS-UPENN EDM Data Mining Challenge" required contestants to predict efficient testtaking based on log data. In this paper, we describe our theory-driven and psychometric modeling approach. For feature engineering, we employed the Log-Normal Response Time Model for estimating latent person speed, and the Generalized…

To support continuous professional development model in the teaching and learning of mathematics, many districts and schools have begun hiring elementary mathematics coaches and/or specialists (MCSs). However, limited large-scale empirical research exists that determines how the use of MCSs affect student learning and achievement. Kristin E.…

This article reviews how National Assessment of Educational Progress (NAEP) has come to collect and analyze data about cognitive and behavioral processes (process data) in the transition to digital assessment technologies over the past two decades. An ordered five-level structure is proposed for describing the uses of process data. The levels in…

State education agencies are sitting on rich repositories of quantitative and qualitative assessment data. This document is designed to provide a conceptual framework and implementation guidance that can help agency leadership leverage and interrogate student performance data in systematic ways for reporting, outreach, and planning purposes. The…

From 2010 onwards, most US states have aligned their education standards by adopting the Common Core State Standards (CCSS) for math and English Language Arts. The CCSS did not target other subjects such as science and social studies. We estimate spillovers of the CCSS on student achievement in non-targeted subjects in models with state and year…

The NAEP EDM Competition required participants to predict efficient test-taking behavior based on log data. This paper describes our top-down approach for engineering features by means of psychometric modeling, aiming at machine learning for the predictive classification task. For feature engineering, we employed, among others, the Log-Normal…

Automatic machine learning (AutoML) methods automate the time-consuming, feature-engineering process so that researchers produce accurate student models more quickly and easily. In this paper, we compare two AutoML feature engineering methods in the context of the National Assessment of Educational Progress (NAEP) data mining competition. The…

In March 2020, the National Center for Special Education Research (NCSER) virtually convened a group of experts to discuss gaps in research on eighth grade mathematics for students with disabilities (SWD) that could be addressed with the NAEP process data and data science techniques for this research. Invited experts also provided recommendations…

The multidimensional item response theory (MIRT) models with covariates proposed by Haberman and implemented in the "mirt" program provide a flexible way to analyze data based on item response theory. In this report, we discuss applications of the MIRT models with covariates to longitudinal test data to measure skill differences at the…

Education has undergone a sweeping renovation throughout the last several decades as part of the school accountability movement aimed to increase student success. High school graduation rates are the highest they have been in decades. School accountability measures continue to be implemented and modified with a goal of increasing student success…

"Mathematical Thinking: From Assessment Items to Challenging Tasks" is a compilation of 36 problem-based lessons that encourage students to engage in productive struggle and deep thinking. Its 36 full-length lessons for grades 2-8 are each inspired by an actual test item from the National Assessment of Educational Progress (NAEP).…

The National Council of Teachers of Mathematics (NCTM) continues to emphasize the importance of early statistical learning; data analysis and probability was the Council's professional development "Focus of the Year" for 2007-2008. Such a focus is needed, especially given the results of the statistics items from the 2003 NAEP. As…

This study examined the 4th-grade National Assessment of Educational Progress (NAEP) assessments (history, geography, and civics) and 3rd- and 4th-grade social studies standards from nine randomly selected states and organized the content around three global concepts (conflict, movement, discovery) and six specific concepts (war, rebellion,…