ERIC Number: ED670912
Record Type: Non-Journal
Publication Date: 2019-May
Pages: 52
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: 0000-00-00
Effect Sizes for Measuring Student and School Growth in Achievement: In Search of Practical Significance. EdWorkingPaper No. 19-60
James Soland; Yeow Meng Thum
Annenberg Institute for School Reform at Brown University
Effect sizes in the Cohen's d family are often used in education to compare estimates across studies, measures, and sample sizes. For example, effect sizes are used to compare gains in achievement students make over time, either in pre- and post-treatment studies or in the absence of intervention, such as when estimating achievement gaps. However, despite extensive research dating back to the paired t-test literature showing that such growth effect sizes should account for within-person correlations of scores over time, such achievement gains are often standardized relative to the standard deviation from a single timepoint or two timepoints pooled. Such a tendency likely occurs in part because there are not many large datasets from which a distribution of student- or school-level gains can be derived. In this study, we present a novel model for estimating student growth in conjunction with a national dataset to show that effect size estimates for student and school growth are often quite different when standardized relative to a distribution of gains rather than static achievement. In particular, we provide nationally representative empirical benchmarks for student achievement and gains, including for male-female gaps in those gains, and examine the sensitivity of those effect sizes to how they are standardized. Our results suggest that effect sizes scaled relative to a distribution of gains are less likely to understate the effects of interventions over time, and that resultant effect sizes often more closely match the estimand of interest for most practice, policy, and evaluation questions.
Descriptors: Effect Size, Academic Achievement, Achievement Gains, Gender Differences, Benchmarking, Computation, Models, Achievement Gap, Achievement Tests, Scores, Grade 3, Grade 8, School Effectiveness, Taxonomy
Annenberg Institute for School Reform at Brown University. Brown University Box 1985, Providence, RI 02912. Tel: 401-863-7990; Fax: 401-863-1290; e-mail: annenberg@brown.edu; Web site: https://annenberg.brown.edu/
Publication Type: Reports - Research
Education Level: Early Childhood Education; Elementary Education; Grade 3; Primary Education; Grade 8; Junior High Schools; Middle Schools; Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Annenberg Institute for School Reform at Brown University
Identifiers - Assessments and Surveys: Measures of Academic Progress
Grant or Contract Numbers: N/A
Author Affiliations: N/A