Current practice for conducting power analyses in hierarchical trials using survey based ICC and effect size estimates may be misestimating power because ICCs are not being adjusted to account for treatment effect heterogeneity. Results presented in Table 1 show that the necessary adjustments can be quite large or quite small. Furthermore, power…

Randomized experiments are generally considered to provide the strongest basis for causal inferences about cause and effect. Consequently randomized field trials have been increasingly used to evaluate the effects of education interventions, products, and services. Populations of interest in education are often hierarchically structured (such as…

Background: Although school climate is linked with youth educational, socioemotional, behavioral, and health outcomes, there has been limited research on the association between school climate and mental health education efforts. We explored whether school climate was associated with students' depression literacy and mental health stigma beliefs.…

Based on a one-year longitudinal experimental study with 3,782 kindergarten school children across 15 countries, this article examines the association between prayer and happiness. Results show that the post-test scores on the faces scale were higher for the participant group who had taken the prayer lessons vis-à-vis the comparison group.…

Research Findings: Mental state verbs (MSV), a component of literate and academic language, may facilitate vocabulary growth, as they relate to metacognitive and metalinguistic awareness as well as decontextualized talk, all of which have been associated with vocabulary growth. In this study, we examined teacher MSV use in group content…

Research Findings: This study examined the potential impacts of ongoing participation (twice weekly for 30 weeks) in teacher-child managed whole-group language and literacy instruction on prekindergarten children's social interaction with classmates. Teacher-child managed whole-group instruction that provides children with opportunities to engage…

Little is known empirically about intraclass correlations (ICCs) for multisite cluster randomized trial (MSCRT) designs, particularly in science education. In this study, ICCs suitable for science achievement studies using a three-level (students in schools in districts) MSCRT design that block on district are estimated and examined. Estimates of…

The purpose of this replication study was to examine the impact of the Teacher Study Group (TSG) professional development in vocabulary on first-grade teachers' knowledge of vocabulary instruction and observed teaching practice, and on students' vocabulary knowledge. Sixty-two schools from 16 districts in four states were randomly assigned to…

Experimental evaluations that involve the educational system usually involve a hierarchical structure (students are nested within classrooms that are nested within schools, etc.). Concerns about contamination, where research subjects receive certain features of an intervention intended for subjects in a different experimental group, have often led…

I discuss three common practices that obfuscate or invalidate the statistical analysis of randomized controlled interventions in applied linguistics. These are (a) checking whether randomization produced groups that are balanced on a number of possibly relevant covariates, (b) using repeated measures ANOVA to analyze pretest-posttest designs, and…

A cluster-randomized trial (CRT) relies on random assignment of intact clusters to treatment conditions, such as classrooms or schools (Raudenbush & Bryk, 2002). One specific type of CRT, a multi-site CRT (MSCRT), is commonly employed in educational research and evaluation studies (Spybrook & Raudenbush, 2009; Spybrook, 2014; Bloom,…

There is growing interest in the use of a multi-tiered system of supports framework to address issues related to school climate and bullying. Positive Behavioral Interventions and Supports (PBIS) is one such model that has received considerable attention; however, nearly all of the extant literature has focused on elementary and middle schools,…

We investigated methods of including covariates in two-level models for cluster randomized trials to increase power to detect the treatment effect. We compared multilevel models that included either an observed cluster mean or a latent cluster mean as a covariate, as well as the effect of including Level 1 deviation scores in the model. A Monte…

As part of the 2010 economic stimulus, a $55 million "Investing in Innovation" (i3) grant from the US Department of Education was awarded to scale up Reading Recovery across the nation. This paper presents the final round of results from the large-scale, mixed methods randomized evaluation of the implementation and impacts of Reading…

Research Findings: This study examined the contribution of several classroom experience measures (classroom characteristics, teacher characteristics, and teacher-child interactions) to preschoolers' improvement in visuomotor integration. Children (N = 467) ranged in age from 3 to 5 years old and were enrolled in 115 classrooms in 5 U.S. states.…