Researchers are often interested in testing the effectiveness of an intervention on multiple outcomes, for multiple subgroups, at multiple points in time, or across multiple treatment groups. The resulting multiplicity of statistical hypothesis tests can lead to spurious findings of effects. Multiple testing procedures (MTPs) are statistical…

Researchers are often interested in testing the effectiveness of an intervention on multiple outcomes, for multiple subgroups, at multiple points in time, or across multiple treatment groups. The resulting multiplicity of statistical hypothesis tests can lead to spurious findings of effects. Multiple testing procedures (MTPs) are statistical…

The present paper, which is intended for a diverse audience of evaluation researchers, applied social scientists, and research funders, provides a broad overview of the conceptual and statistical issues involved in using multisite randomized trials to learn "about" and "from" variation in program effects across…

In education research and in many other fields, researchers are often interested in testing the effectiveness of an intervention on multiple outcomes, for multiple subgroups, at multiple points in time, or across multiple treatment groups. The resulting multiplicity of statistical hypothesis tests can lead to spurious findings of effects. Multiple…

Suppose an education researcher wants to test the impact of a high school drop-out prevention intervention in which at-risk students attend classes to receive intensive summer school instruction. The district will allow the researcher to randomly assign students to the treatment classes or to the control group. Half of the students (the treatment…

Selection bias is problematic when evaluating the effects of postsecondary interventions on college students, and can lead to biased estimates of program effects. While instrumental variables can be used to account for endogeneity due to self-selection, current practice requires that all five assumptions of instrumental variables be met in order…

This report summarizes the complex research literature on quantitative methods for assessing how impacts of educational interventions on instructional practices and student learning differ across students, educators, and schools. It also provides technical guidance about the use and interpretation of these methods. The research topics addressed…

Supplemental Educational Services (SES), a component of the No Child Left Behind (NCLB) Act, provides free tutoring to economically disadvantaged children who are attending Title I schools in their 2nd or more years of school improvement. This research evaluated SES in Tennessee to determine the: (a) impacts on student achievement, and (b)…

Community coalitions (CCs) have labored with some difficulty to demonstrate empirical evidence of effectiveness in preventing a wide range of adolescent problem behaviors. Training and technical assistance (TA) have been identified as important elements in promoting improved functioning of CCs. A reliable, valid, and inexpensive method to assess…

This paper compares five growth measures using the High School and Beyond database. The measures are: (1) simple gain (posttest minus pretest); (2) difference between group means; (3) percentage of students who scored higher on the posttest than the pretest; (4) percentage of items missed on the pretest which are subsequently answered correctly on…

This paper argues that inferential statistical analyses as part of group research designs do not provide the kinds of data that are useful in determining the efficacy of clinical intervention with individuals having communication disorders. Replication studies are seen as providing a more fruitful approach to ascertaining the efficacy of clinical…

This paper encourages the use of multiple logistic analysis in early intervention research, to assess the degree of association of multiple factors (such as subject or situational characteristics) with a dichotomous outcome (such as benefitting or not benefitting from an intervention) and to estimate the probability of each outcome. (JDD)

The goal of college preparation programs is to help students, particularly underrepresented students of color, to enroll and succeed in college. While many programs focus on strengthening students' math and reading skills, familiarizing them with college entrance exams, and demystifying the college admissions process, there are few guidelines for…

Mixed methods research has increased in popularity over the past 20 years. Literature reveals that exploratory qualitative analysis followed by confirmatory survey research is common and concurrent studies outnumber longitudinal design. Longitudinal studies using quantitative and qualitative methods in sequence for exploratory purposes are rare,…

This paper reviews analytic methods for estimating growth curves or developmental trajectories from longitudinal data. Traditional models for studying change are contrasted with a relatively new method, hierarchical linear models. Evidence is presented indicating that hierarchical linear models can overcome some of the methodological limitations…