This article presents the Renger System Test (RST) as a method for assessing whether a system evaluation approach is suitable for evaluating complex interventions. The RST has three criteria: (1) the intervention includes multiple components, (2) these components operate interdependently, and (3) their interdependence produces an outcome that no…

The use of complexity-based evaluation methods remains relatively underexplored in the field of evaluation. While increasingly employed to assess complex interventions, Contribution Analysis (CA) continues to suffer from a lack of operationalization. In this article, we propose enhancing the implementation of CA by leveraging Participatory Systems…

This study aimed to assess the methods and outcomes of The Measurement Tool to Assess systematic Reviews (AMSTAR) 2 appraisals in overviews of reviews (overviews) of interventions in the cardiovascular field and identify factors that are associated with these outcomes. MEDLINE, Scopus, and the Cochrane Database of Systematic Reviews were searched…

The theoretical defensibility of applied linguistic intervention design provides the rationale for such designs, demonstrating how they are supported by theory, constructs, and analysis. Though founded upon science, imaginative design has its leading technical function as guiding lodestar. A scientific rationale for a plan underwrites the design,…

This dissertation used synthetic datasets, semi-synthetic datasets, and a real-world dataset from an educational intervention to compare the performance of 15 machine learning and multiple imputation methods to estimate the individual treatment effect (ITE). In addition, it examined the performance of five evaluation metrics that can be used to…

Selection and implementation of evidence-based reading interventions is a complex process that requires careful consideration of factors such as intervention effectiveness and social validity. Social validity refers to how key stakeholders, including teachers, students, and parents, perceive the goals, methods, and outcomes of an intervention.…

The improvement of existing science education interventions and the development of new ones requires cohesive, large-scale evaluation and testing that inform what works, for whom, and in what context. Systems approaches to design, implement, and evaluate interventions require knowledge, skills, and expertise in studying humans alongside the lived…

Measures of learning speed (i.e., the amount of learning/cumulative time learner spends engaged in an intervention) are rarely included in research designed to evaluate and compare academic interventions. We build a case that includes analyses of learning speed metrics in academic intervention research can provide more useful information for…

The number of real-world laboratories (RwLs) as research settings to address sustainability problems by using collaborative and experimental approaches has been growing steadily over the past years. RwLs are widely considered promising settings for the production of action-orientated knowledge in order to contribute solving wicked real-world…

When teachers work with students exhibiting academic failure, they may look to factors outside of instruction such as a student's home life or perceived disability as explanations. Placing the locus of control outside of the instructional context becomes a convenient way to escape culpability for unsatisfactory outcomes. A more functional approach…

Computational Thinking (CT) is considered a core 21st century digital skill. The aspect of assessment is crucial and knowing what, who, when, how, and where to assess is important for assessment design. In this study, we conducted an umbrella review to gain insights regarding CT assessment in higher education. In total, we analyzed 11 reviews,…

Feedback is a key factor for learning success and has therefore been widely studied in higher education. As feedback is a highly contextualized practice serving various learner needs, researchers have utilized a plethora of feedback designs in their intervention studies. This diversity in feedback conceptualizations and pedagogical designs often…

Background: With the right support, people with learning disabilities can be 'good enough' parents (Coren et al., 2011; Murphy & Feldman, 2002). Parenting programmes exist to support parents and are made accessible for people with learning disabilities who are expecting a baby, but evaluation of the benefit of these interventions is poor due…

The visual analysis of data presented in time-series graphs are common in single-case design (SCD) research and applied practice in school psychology. A growing body of research suggests that visual analysts' ratings are often influenced by construct-irrelevant features including Y-axis truncation and compression of the number of data points per…

The tools charts are some of the most frequently visited resources on the NCII website. Furthermore, current and prior federal legislation and increasingly state legislation, emphasizes the use of educational programs, interventions, and assessments that have an evidence base supporting their effectiveness. State education agencies (SEAs) play an…