Multisite field experiments using the (generalized) randomized block design that assign treatments to individuals within sites are common in education and the social sciences. Under this design, there are two possible estimands of interest and they differ based on whether sites or blocks have fixed or random effects. When the average treatment…

Randomized experiments involving education interventions are typically implemented as cluster randomized trials, with schools serving as clusters. To design such a study, it is critical to understand the degree to which learning outcomes vary between versus within clusters (schools), specifically the intraclass correlation coefficient. It is also…

Education experiments frequently assign students to treatment or control conditions within schools. Longitudinal components added in these studies (e.g., students followed over time) allow researchers to assess treatment effects in average rates of change (e.g., linear or quadratic). We provide methods for a priori power analysis in three-level…

For decades, the notion of teachers engaging in research has been discussed and carried out under the heuristics and methodologies of action research (Manfra, 2019; Pine, 2009). A typical action research project might involve an individual teacher studying the effectiveness of a specific instructional strategy like having students preview content…

To identify the ways teachers and educational systems can improve learning, researchers need to make causal inferences. Analyses of existing datasets play an important role in detecting causal patterns, but conducting experiments also plays an indispensable role in this research. In this article, we advocate for experiments to be embedded in real…

Questions regarding the generalizability of results from educational experiments have been at the forefront of methods development over the past five years. This work has focused on methods for estimating the effect of an intervention in a well-defined inference population (e.g., Tipton, 2013; O'Muircheartaigh and Hedges, 2014); methods for…

This article describes a predictive model that assesses whether a student will have greater perceived learning in group assignments or in individual work. The model produces correct classifications 87.5% of the time. The research is notable in that it is the first in the education literature to adopt a predictive modeling methodology using data…

Experimental evaluations that involve the educational system usually involve a hierarchical structure (students are nested within classrooms that are nested within schools, etc.). Concerns about contamination, where research subjects receive certain features of an intervention intended for subjects in a different experimental group, have often led…

Education planning provides the policy maker and the decision maker a logical framework in which to develop and implement education policy. At the international level, education planning is often confounded by both internal and external complexities, making the development of education policy difficult. This research presents a discrete event…

As evidence-based reform becomes increasingly important in educational policy, it is becoming essential to understand how research design might contribute to reported effect sizes in experiments evaluating educational programs. The purpose of this study was to examine how methodological features such as types of publication, sample sizes, and…

While split-plot designs have received considerable attention in the literature over the past decade, there seems to be a general lack of intuitive understanding of the error structure of these designs and the resulting statistical analysis. Typically, students learn the proper error terms for testing factors of a split-plot design via "expected…

The meaning of fractions with integer multiplication is something that is difficult to understand by students. They tend to think that the product it produces a larger number, while the multiplication of fractions with integers, the result can be any number larger or smaller. This study is a research design that aims to develop a local…

Due to the clustered nature of field data, multi-level modeling has become commonly used to analyze data arising from educational field experiments. While recent methodological literature has focused on multi-level mediation analysis, relatively little attention has been devoted to mediation analysis when three levels (e.g., student, class,…

As the nation witnesses a distinct decline in civic engagement among young adults, political science instructors across the nation face the formidable task of engaging students in lower-level, general education courses outside students' primary domain of interest. The research presented here seeks to understand if visually enhanced lecture…

This article gives an example of how student-conducted experiments can enhance a course in the design of experiments. We focus on a project whose aim is to find a good mixture of water, soap and glycerin for making soap bubbles. This project is relatively straightforward to implement and understand. At its most basic level the project introduces…