State education agencies are sitting on rich repositories of quantitative and qualitative assessment data. This document is designed to provide a conceptual framework and implementation guidance that can help agency leadership leverage and interrogate student performance data in systematic ways for reporting, outreach, and planning purposes. The…

This article reviews how National Assessment of Educational Progress (NAEP) has come to collect and analyze data about cognitive and behavioral processes (process data) in the transition to digital assessment technologies over the past two decades. An ordered five-level structure is proposed for describing the uses of process data. The levels in…

Although the number of countries conducting large-scale assessments has significantly increased over the past two decades, this has not necessarily led to the effective use of learning assessment data in policymaking and planning. To better understand the reasons for this, the UNESCO International Institute for Educational Planning (IIEP)…

The NAEP EDM Competition required participants to predict efficient test-taking behavior based on log data. This paper describes our top-down approach for engineering features by means of psychometric modeling, aiming at machine learning for the predictive classification task. For feature engineering, we employed, among others, the Log-Normal…

Although the number of countries conducting large-scale assessments has increased significantly over the past two decades, this has not necessarily led to the effective use of learning assessment data in policy-making and planning. To better understand the reasons for this, the UNESCO International Institute for Educational Planning (IIEP)…

The standard methodology for many large-scale assessments in education involves regressing latent variables on numerous contextual variables to estimate proficiency distributions. To reduce the number of contextual variables used in the regression and improve estimation, we propose and evaluate principal component analysis on the covariance matrix…

Drawing upon recent theorising of numbers and data, and applications to schooling, this paper reveals how tensions between more accountability-oriented logics, and more contextually-situated conceptions of engagement with data, played out in one school in a regional community in northern Queensland, Australia. The research reveals that at the same…

This study examines the impact of early Algebra I coursework on advanced Carnegie credits among 12th graders from diverse racial/ethnic backgrounds, using data from the NCES HSTS (1990-2019). Findings indicate that early Algebra students, particularly Black and Latinx, earn more advanced credits, revealing a widening gap in advanced course…

The "2nd Annual WPI-UMASS-UPENN EDM Data Mining Challenge" required contestants to predict efficient testtaking based on log data. In this paper, we describe our theory-driven and psychometric modeling approach. For feature engineering, we employed the Log-Normal Response Time Model for estimating latent person speed, and the Generalized…

The technological capacity that now exists to gather and store large amounts of data has improved access to information related to the academic performance, demographic profile and welfare needs of young people in a school. Improvements in data management techniques, statistical data analysis and software packages have made retrieving and…

To support continuous professional development model in the teaching and learning of mathematics, many districts and schools have begun hiring elementary mathematics coaches and/or specialists (MCSs). However, limited large-scale empirical research exists that determines how the use of MCSs affect student learning and achievement. Kristin E.…

From 2010 onwards, most US states have aligned their education standards by adopting the Common Core State Standards (CCSS) for math and English Language Arts. The CCSS did not target other subjects such as science and social studies. We estimate spillovers of the CCSS on student achievement in non-targeted subjects in models with state and year…

Automatic machine learning (AutoML) methods automate the time-consuming, feature-engineering process so that researchers produce accurate student models more quickly and easily. In this paper, we compare two AutoML feature engineering methods in the context of the National Assessment of Educational Progress (NAEP) data mining competition. The…

In March 2020, the National Center for Special Education Research (NCSER) virtually convened a group of experts to discuss gaps in research on eighth grade mathematics for students with disabilities (SWD) that could be addressed with the NAEP process data and data science techniques for this research. Invited experts also provided recommendations…

Statistical terms are used in everyday language and, at times, used in non-statistical ways. It is often assumed students understand statistical terms because of their common use; however, research into their understanding of specific statistical terms is scant. This report focuses on 58 Year 3 students' responses to the basic question, "What…