NotesFAQContact Us
Collection
Advanced
Search Tips
Showing all 7 results Save | Export
The Effect-Size Benchmark That Matters Most: Education Interventions Often Fail
Peer reviewed Peer reviewed
Direct linkDirect link
Kraft, Matthew A. – Educational Researcher, 2023
It is a healthy exercise to debate the merits of using effect-size benchmarks to interpret research findings. However, these debates obscure a more central insight that emerges from empirical distributions of effect-size estimates in the literature: Efforts to improve education often fail to move the needle. I find that 36% of effect sizes from…
Descriptors: Effect Size, Benchmarking, Educational Research, Educational Policy
Using Pupils' Grade Obtained in National Examinations as an Outcome Measure in Evaluations: Some Considerations for the Design of Randomised Controlled Trials
Peer reviewed Peer reviewed
Direct linkDirect link
Ben Smith; Stephen P. Morris; Harry Armitage – Research Papers in Education, 2023
It is not uncommon for randomised trials in education to have the performance of sample members in national examinations as their primary outcome. In many cases, examination results are available as summary measures only. Taking the example of GCSE examination results in England, this paper shows that using summary measures of an underlying score…
Descriptors: Language Arts, Mathematics Instruction, Grades (Scholastic), Standardized Tests
Power Calculations for Randomized Controlled Trials with Auxiliary Observational Data
Peer reviewed Peer reviewed
PDF on ERIC Download full text
Jaylin Lowe; Charlotte Z. Mann; Jiaying Wang; Adam Sales; Johann A. Gagnon-Bartsch – International Educational Data Mining Society, 2024
Recent methods have sought to improve precision in randomized controlled trials (RCTs) by utilizing data from large observational datasets for covariate adjustment. For example, consider an RCT aimed at evaluating a new algebra curriculum, in which a few dozen schools are randomly assigned to treatment (new curriculum) or control (standard…
Descriptors: Randomized Controlled Trials, Statistical Analysis, Middle School Students, High School Students
Transforming Assessments of Clinician Knowledge: A Randomized Controlled Trial Comparing Traditional Standardized and Longitudinal Assessment Modalities
Peer reviewed Peer reviewed
PDF on ERIC Download full text
Shahid A. Choudhry; Timothy J. Muckle; Christopher J. Gill; Rajat Chadha; Magnus Urosev; Matt Ferris; John C. Preston – Practical Assessment, Research & Evaluation, 2024
The National Board of Certification and Recertification for Nurse Anesthetists (NBCRNA) conducted a one-year research study comparing performance on the traditional continued professional certification assessment, administered at a test center or online with remote proctoring, to a longitudinal assessment that required answering quarterly…
Descriptors: Nurses, Certification, Licensing Examinations (Professions), Computer Assisted Testing
Using Publicly Available Auxiliary Data to Improve Precision of Treatment Effect Estimation in a Randomized Efficacy Trial
Peer reviewed Peer reviewed
PDF on ERIC Download full text
Direct linkDirect link
Charlotte Z. Mann; Jiaying Wang; Adam Sales; Johann A. Gagnon-Bartsch – Grantee Submission, 2024
The gold-standard for evaluating the effect of an educational intervention on student outcomes is running a randomized controlled trial (RCT). However, RCTs may often be small due to logistical considerations, and resulting treatment effect estimates may lack precision. Recent methods improve experimental precision by incorporating information…
Descriptors: Intervention, Outcomes of Education, Randomized Controlled Trials, Data Use
Using Publicly Available Auxiliary Data to Improve Precision of Treatment Effect Estimation in a Randomized Efficacy Trial
Peer reviewed Peer reviewed
PDF on ERIC Download full text
Charlotte Z. Mann; Jiaying Wang; Adam Sales; Johann A. Gagnon-Bartsch – International Educational Data Mining Society, 2024
The gold-standard for evaluating the effect of an educational intervention on student outcomes is running a randomized controlled trial (RCT). However, RCTs may often be small due to logistical considerations, and resulting treatment effect estimates may lack precision. Recent methods improve experimental precision by incorporating information…
Descriptors: Intervention, Outcomes of Education, Randomized Controlled Trials, Data Use
Power Calculations for Randomized Controlled Trials with Auxiliary Observational Data
Peer reviewed Peer reviewed
Direct linkDirect link
Jaylin Lowe; Charlotte Z. Mann; Jiaying Wang; Adam Sales; Johann A. Gagnon-Bartsch – Grantee Submission, 2024
Recent methods have sought to improve precision in randomized controlled trials (RCTs) by utilizing data from large observational datasets for covariate adjustment. For example, consider an RCT aimed at evaluating a new algebra curriculum, in which a few dozen schools are randomly assigned to treatment (new curriculum) or control (standard…
Descriptors: Randomized Controlled Trials, Middle School Mathematics, Middle School Students, Middle Schools