Data extraction is a crucial, yet labor-intensive and error-prone part of evidence synthesis. To date, efforts to harness machine learning for enhancing efficiency of the data extraction process have fallen short of achieving sufficient accuracy and usability. With the release of large language models (LLMs), new possibilities have emerged to…

Background: The data revolution in education has led to more data collection, more randomized controlled trials (RCTs), and more data collection within RCTs. Often following IES recommendations, researchers studying program effectiveness gather data on how the intervention was implemented. Educational implementation data can be complex, including…

Randomized experiments involving education interventions are typically implemented as cluster randomized trials, with schools serving as clusters. To design such a study, it is critical to understand the degree to which learning outcomes vary between versus within clusters (schools), specifically the intraclass correlation coefficient. It is also…

Randomized A/B tests within online learning platforms represent an exciting direction in learning sciences. With minimal assumptions, they allow causal effect estimation without confounding bias and exact statistical inference even in small samples. However, often experimental samples and/or treatment effects are small, A/B tests are underpowered,…

The current situation due to the COVID-19 pandemic has had huge consequences in all aspects of our lives, including the development of research activities. Studies involving face-to-face interaction with people, such as randomized controlled trials, were the first affected. The objective of this article is to present the insights on challenges…

Recent methods have sought to improve precision in randomized controlled trials (RCTs) by utilizing data from large observational datasets for covariate adjustment. For example, consider an RCT aimed at evaluating a new algebra curriculum, in which a few dozen schools are randomly assigned to treatment (new curriculum) or control (standard…

In the coming years, public programs will capture even more and richer data than they do now, including data from web-based tools used by participants in employment services, from tablet-based educational curricula, and from electronic health records for Medicaid beneficiaries. Program evaluators seeking to take full advantage of these data…

Low- and middle-income countries have made significant progress getting children into school, but student learning and achievement are often dreadfully low (Berry, Barnett, & Hinton, 2015; Pritchett, 2013). Approximately 250 million children across the world are not acquiring basic reading and math skills, even though about half have spent at…

The Rapid Assessment of Cognitive and Emotional Regulation (RACER) is a tablet-based assessment tool for children that measures executive function (EF) skills. Instructions that are brief and visually presented; game-like tasks are designed to easily engage children regardless of literacy level and variable test administration settings. RACER…

Qualitative researchers often have to decide whether to collect data using focus groups or individual interviews. We systematically compare these two methods on their ability to generate two types of information: unique items in a brainstorming task and personally sensitive disclosures. Our study sample consisted of 350 African-American men living…

Purpose/Objective: Under the reauthorized Every Student Succeeds Act (ESSA), states and districts retain greater discretion over the measures included in school quality report cards. Moreover, ESSA now requires states to expand their measurement efforts to address factors like school climate. This shift toward more comprehensive measures of school…

National governments use evidence selectively to argue that a successful policy intervention in one local area should be emulated in others ("evidence-based best practice"). However, the value of such evidence is always limited because there is: disagreement on the best way to gather evidence of policy success, uncertainty regarding the…

Designing, developing, and implementing a multi-year randomized control trial targeting what works in gifted education mathematics study is challenging, at best. In response to the United States Department of Education's Institute of Education Sciences request for proposal, we carefully constructed a mixed-methods study by assembling a diverse…

Background: An Internet-based relapse prevention supplement to adolescent substance abuse treatment programming is a promising modality to reinforce treatment gains and enhance recovery; however, an evidence base is lacking. Objective: To assess the efficacy of the online Navigating my Journey (NmJ) program. Methods: 129 adolescent-aged…

With the increase in the use of online professional development resources, especially for rural educators, this study evaluated the effects of online training modules on in-service teachers' ability to collect and use instructional progress monitoring data to make instructional decisions for students with significant disability (i.e., moderate to…