ERIC Number: ED663567
Record Type: Non-Journal
Publication Date: 2024-Sep-21
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
Participatory Methods in Randomized Control Trials: Exploring Current Practices and Future Direction to Advance Equity in Educational Research
Wendy Castillo; Lindsay Dusard
Society for Research on Educational Effectiveness
Background: The emergence of causal research in education was almost strictly quantitative twenty years ago, however, that landscape has changed considerably. The number of intervention studies fielded and completed annually has increased substantially, and the quality of the evaluations is much more robust, including paying much greater attention to qualitative aspects such as, context, implementation, and heterogeneity of effectiveness (National Academies of Sciences, Engineering, and Medicine, 2022). To date, Institute of Education Sciences (IES) funded over 100 RCT evaluations through the National Centers for Education Research and for Special Education Research (NCER and NCSER, respectively) (see https://ies.ed.gov/funding/grantsearch/index.asp). Randomized control trials (RCTs) are known to be at odds with the values of many within the global majority (Rink et al., 2020, Macaulay, 2017, Shahjahan, 2011). There has been criticism of RCTs as they have excluded a racial equity perspective (Castillo & Maynard, 2023) and can be exploitative for the groups we as a field say we care most about. Despite this reality, randomized control trials are still considered a standard within the field of educational research (What Works Clearinghouse, 2022). Rightfully so, a central focus of today's educational research is advancing equity. Since RCTs still are some of the most powerful drivers of policy change and investment, we should consider alternative approaches by turning to critical frameworks that honor the knowledge and values of the community while pursuing rigorous causal evidence that matters. Focus of the Study: This paper aims to improve the quality of causal inference research produced in education by systematically reviewing current practices and future opportunities to incorporate participatory and racially focused methods in RCTs. Participatory research (used here as an umbrella term for participatory frameworks is primarily concerned with breaking down power dynamics between the researchers and researched (Macaulay, 2017). It assigns equal value to knowledge and experiences held by all participating members. The review is focused on RCTs for two reasons 1) RCTs have been long recognized in the scientific community as the gold standard for causal inference research and 2) RCTs typically have more flexibility to include community voices and collect new data (including racial/ethnic data and new measures centered on community values) for new analyses compared to quasi-experimental designs as it involves primary versus post-hoc research. Specifically, the study addresses three questions: 1.) To what extent have participatory research methods (broadly defined) and/or racially-focused been used in randomized control trials in education and related fields? 2.) What were the central motivating factors for and perceived benefits of incorporating participatory and racially focused methods in randomized control trials? 3.) How have other fields used participatory methods in randomized control trials? What practices align with the context and dynamics of educational research? Research Design: To address these questions, a mixed methods study using an exploratory sequential design is employed. The first strand of inquiry relies on a systematic review of peer-reviewed and gray literature. Studies are coded based on their expressed use of participatory methods, population of interest, who was involved, and to what extent they were involved. Informed by the first, the second strand seeks to gain a more nuanced understanding of the use of participatory methods within randomized control trials. Using purposive, nested sampling, interviews are conducted with authors (including researchers, community partners, and participants) to understand the experience and perceived benefits. Qualitative latent and manifest content analysis are used to explore common themes, both explicit and implicit (Kleinheksel, 2020). Finally, we review practices from other fields. In particular, we draw inspiration from systematic reviews of participatory clinical trials in medicine to explore the opportunities and potential for this work (McFarlane, 2022). And we evaluate their potential impact using the Participatory Justice framework (Whiteford et al, 2018). Results and Conclusions: Initial findings show very few studies in education, however in public health and social services, we found this approach in five examples thus far see Table 1. The most commonly involved community members were administrators/service providers and most often at the study design phase of the research process. We will formalize our search approach and begin work toward the second and third research questions, with work expected to conclude by July 2024.
Descriptors: Randomized Controlled Trials, Educational Research, Equal Education, Educational Policy, Causal Models, Evidence, Statistical Inference, Participatory Research
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Grant or Contract Numbers: N/A
Author Affiliations: N/A