This article presents the Renger System Test (RST) as a method for assessing whether a system evaluation approach is suitable for evaluating complex interventions. The RST has three criteria: (1) the intervention includes multiple components, (2) these components operate interdependently, and (3) their interdependence produces an outcome that no…

The use of complexity-based evaluation methods remains relatively underexplored in the field of evaluation. While increasingly employed to assess complex interventions, Contribution Analysis (CA) continues to suffer from a lack of operationalization. In this article, we propose enhancing the implementation of CA by leveraging Participatory Systems…

Intended for researchers and clinical leaders, this article suggests that embedded program evaluation is a good fit with the desired features of practice-oriented research. The systematic nature of evaluation that is built into the operational workflow of a practice setting may increase the diversity of methods available to explore processes and…

The Region 5 Comprehensive Center (R5CC) developed the Education Agency State of Readiness Assessment Tool to assess the readiness of State education agencies (SEA) to take on capacity-building projects. The indicators are adapted from the NECTAC Technical Assistance Model for Long-Term Systems Change and are based on years of developing and…

The Campbellian validity typology has been used as a foundation for outcome evaluation and for developing evidence-based interventions for decades. As such, randomized control trials were preferred for outcome evaluation. However, some evaluators disagree with the validity typology's argument that randomized controlled trials as the best design…

Using social constructivist theories of adult learning, the Faculty Pedagogical Development (PD) Programme was designed to offer faculty a unique professional learning experience at one university in Qatar. This study reports on the evaluation of this PD programme that involved 24 participants from different colleges in its first round of…

The System Transformation for Equitable Preschools (STEP Forward with Data) Framework was developed to help preschool systems leaders understand where in the preschool system there may be inequities in how children and families are accessing, being supported by, and experiencing the system. The Framework offers preschool systems leaders a pathway…

Distance learning technology continues to grow and undergo significant changes in higher education settings. Applying strategic planning methods from organizational behavior and operations management can act as a useful guide for the implementation of a sustainable distance learning program. Strong planning can help ensure the distance learning…

The Region 15 Comprehensive Center (R15CC) is federally funded to provide high-impact, customized, and responsive capacity-building technical assistance to state educational agencies (SEAs) in Arizona, California, Nevada, and Utah. In 2022, R15CC established a regional Family and Community Engagement (FACE) Collaborative for SEA leaders in…

In this article, we reflect on our experience applying a framework for evaluating systems change to an evaluation of a statewide West Virginia alliance funded by the National Science Foundation (NSF) to improve the early persistence of rural, first-generation, and other underrepresented minority science, technology, engineering, and mathematics…

The National Academies Global Forum on Innovations in Health Professional Education held a public workshop in November 2023. Speakers explored how health professional educators and providers with health systems science (HSS) training can prepare learners to work together within health systems, resulting in better patient care, improved population…

Although many have suggested the use of games to motivate active learning, studies that evaluate the learning outcomes of games with high school students are scarce. Here, we present the evaluation of the board game "Taphonomy: Dead and Fossilized" as an active learning tool to teach fossilization and Earth systems thinking with rising…

Project EQuIPD--Engaging Quality Instruction through Professional Development--sought to establish and test for efficacy a professional development model to produce highly qualified teachers in STEM practices for all children, especially for students who are in traditionally underserved schools and districts within the State of Florida. The…