Multi-site randomized controlled trials (RCTs) provide unbiased estimates of the average impact in the study sample. However, their ability to accurately predict the impact for individual sites outside the study sample, to inform local policy decisions, is largely unknown. To extend prior research on this question, we analyzed six multi-site RCTs…

Educational researchers advocate the use of an effect size and its confidence interval to assess the effectiveness of interventions instead of relying on a p-value, which has been blamed for lack of reproducibility of research findings and the misuse of statistics. The aim of this study is to provide a framework, which can provide direct evidence…

Awareness of students' learning status, and maintaining students' focus and attention during class are important issues in classroom management. Several observation instruments have been designed for human observers to document students' engagement in class, but the processes are time-consuming and laborious. Recently, with the development of…

We examine the effect of an integrity pilot campaign on undergraduates' behavior. As with many costly small-scale experiments and pilot programs, our statistical inference has to rely on small sample size. To tackle this issue, we perform a Bayesian retrospective power analysis. In our setup, a lecturer intentionally makes mistakes that favors…

These are the appendices for the report "Study of College Transition Messaging in GEAR UP: Impacts on Enrolling and Staying in College." Five appendices are included in the document: (1) Additional Details about College Transition Messaging; (2) Additional Information about How the Study Was Designed and Implemented; (3) Supplemental…

Educational stakeholders are keen to know the magnitude and importance of different interventions. However, the way evidence is communicated to support understanding of the effectiveness of an intervention is controversial. Typically studies in education have used the standardised mean difference as a measure of the impact of interventions. This…

This study reports on what 13 teacher educators going through a procedure to become registered as a teacher educator in 2011-2012 learned, what goals they formulated for their further professional development and what activities they planned to achieve these goals. The methods used in this study are mainly the same as were used at the time the…

This meta-analysis synthesizes the literature on interventions for struggling readers in Grades 4 through 12 published between 1980 and 2011. It updates Scammacca et al.'s analysis of studies published between 1980 and 2004. The combined corpus of 82 study-wise effect sizes was meta-analyzed to determine (a) the overall effectiveness of reading…

Interest in variation in program impacts--How big is it? What might explain it?--has inspired recent work on the analysis of data from multi-site experiments. One critical aspect of this problem involves the use of random or fixed effect estimates to visualize the distribution of impact estimates across a sample of sites. Unfortunately, unless the…

The effects of a mathematical problem-solving intervention on students' problem-solving performance and math achievement were measured in a randomized control trial with 1,059 7th-grade students. The intervention, "Solve It!," is a research-based cognitive strategy instructional intervention that was shown to improve the problem-solving…

Background: Results from MDRC's longitudinal, random-assignment evaluation of career-academy high schools reveal that several years after high-school completion, those randomized to receive the academy opportunity realized a $175 (11%) increase in monthly earnings, on average. Objectives: In this paper, I investigate the impact of duration of…

There are two mainstream curricula for international school students at the junior high level: the International Baccalaureate (IB) Middle Years Programme (MYP) and the Cambridge International General Certificate of Secondary Education (IGCSE). The former was developed in the mid-1990s and is currently being relaunched in a 21st-century approach.…

KIPP (Knowledge is Power Program) is a national network of public charter schools whose stated mission is to help underserved students enroll in and graduate from college. Prior studies (see Tuttle et al. 2013) have consistently found that attending a KIPP middle school positively affects student achievement, but few have addressed longer-term…

The well established, gold standard approach to finding out what works in education research is to run a randomized controlled trial (RCT) using a standard pre-test and post-test design. RCTs have been used in the intelligent tutoring community for decades to determine which questions and tutorial feedback work best. Practically speaking, however,…

In 23 states, members of the high school class of 2008 were required to pass a state high school exit examination (HSEE) to earn regular high school diplomas. Proponents of these policies claim that they improve student academic achievement, although critics argue that they reduce the quality of instruction without raising academic achievement.…