This article offers a case example of how experimental evaluation methods can be coupled with principles of design-based implementation research (DBIR), improvement science (IS), and rapid-cycle evaluation (RCE) methods to provide relatively quick, low-cost, credible assessments of strategies designed to improve programs, policies, or practices.…

Purpose: This study presents a novel methodology/process for assessing inclusion of theoretically-based implementation factors within available adoption-ready health promotion programs. Methods: Classroom-based physical activity (CBPA) programs were used as an example to describe the process. Our team selected an implementation science framework…

Schools are increasingly turning towards restorative practices as a pathway to building schools with stronger relationships, justice, and equity. While effectiveness studies are increasing, too little attention is focused on evaluating implementation. This resources is for audiences who evaluate implementation of whole school restorative practices…

Lottery-based identification strategies offer potential for generating the next generation of evidence on U.S. early education programs. The authors' collaborative network of five research teams applying this design in early education settings and methods experts has identified six challenges that need to be carefully considered in this next…

Given the widespread use of non-experimental (NE) methods for assessing program impacts, there is a strong need to know whether NE approaches yield causally valid results in field settings. In within-study comparison (WSC) designs, the researcher compares treatment effects from an NE with those obtained from a randomized experiment that shares the…

The foster youth education program strives to increase graduation rates by improving academic performance and encouraging student retention through an innovative collaboration between tutors, social workers, and schools to provide individualized support. The pilot program was funded by U.S. Department of Education Innovation Funds, which evolving…

In the present study, we investigated the relative impact of age- versus schooling-related growth in school readiness skills using four modeling approaches that leverage natural variation in longitudinal data collected within the preschool year. Our goal was to demonstrate the applicability of different analytic techniques that do not rely on…

This report from the Career and Technical Education (CTE) Research Network Lead team provides final results from an evaluability assessment of CTE programs that feasibly could be evaluated using a rigorous experimental design. Evaluability assessments (also called feasibility studies) are used in education and other fields, such as international…

Over 60 years of research reveal that informants who observe youth in clinically relevant contexts (e.g., home, school)--typically parents, teachers, and youth clients themselves--often hold discrepant views about that client's needs for mental health services (i.e., "informant discrepancies"). The last 10 years of research reveal that…

We describe here lessons learned in designing an early algebra curriculum to measure early algebra's impact on children's algebra readiness for middle grades. The curriculum was developed to supplement regular mathematics instruction in Grades K-5. Lessons learned centered around the importance of several key factors, including using conceptual…

Engaging, focusing, and persisting in the completion of tasks are among the skills needed for school success. Tracking whether a child is learning cognitive problem-solving skills is essential in knowing if they are acquiring skills important for development and school readiness; and if not, how they are responding to early intervention. Use of…

Tools for evaluating the quality and rigor of single case research designs (SCD) are often used when conducting SCD syntheses. Preferred components include evaluations of design features related to the internal validity of SCD to obtain quality and/or rigor ratings. Three tools for evaluating the quality and rigor of SCD (Council for Exceptional…

Classroom management practices are critical to the success of teachers and students, and a growing number of programs have been developed to improve these practices. However, there has been less investigation into observational tools to assess classroom management and explore whether it can be measured consistently by observers across elementary,…

Varying methods for evaluating the outcomes of single case research designs (SCD) are currently used in reviews and meta-analyses of interventions. Quantitative effect size measures are often presented alongside visual analysis conclusions. Six measures across two classes--overlap measures (percentage non-overlapping data, improvement rate…

This study explores the impact of Strategic and Interactive Writing Instruction (SIWI) on six students' written language skills through the application of a multiple-baseline probe single case design with embedded condition. This was part of a larger Institute of Education Sciences (IES)-funded project focused on the development and feasibility of…