ERIC Number: EJ1291561
Record Type: Journal
Publication Date: 2021-Mar
Pages: 23
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0272-2631
EISSN: N/A
Available Date: N/A
A Call for Cautious Interpretation of Meta-Analytic Reviews
Boers, Frank; Bryfonski, Lara; Faez, Farahnaz; McKay, Todd
Studies in Second Language Acquisition, v43 n1 p2-24 Mar 2021
Meta-analytic reviews collect available empirical studies on a specified domain and calculate the average effect of a factor. Educators as well as researchers exploring a new domain of inquiry may rely on the conclusions from meta-analytic reviews rather than reading multiple primary studies. This article calls for caution in this regard because the outcome of a meta-analysis is determined by how effect sizes are calculated, how factors are defined, and how studies are selected for inclusion. Three recently published meta-analyses are reexamined to illustrate these issues. The first illustrates the risk of conflating effect sizes from studies with different design features; the second illustrates problems with delineating the variable of interest, with implications for cause-effect relations; and the third illustrates the challenge of determining the eligibility of candidate studies. Replication attempts yield outcomes that differ from the three original meta-analyses, suggesting also that conclusions drawn from meta-analyses need to be interpreted cautiously.
Descriptors: Meta Analysis, Literature Reviews, Effect Size, Computation, Selection Criteria, Research Design, Research Problems, Replication (Evaluation)
Cambridge University Press. 100 Brook Hill Drive, West Nyack, NY 10994. Tel: 800-872-7423; Tel: 845-353-7500; Fax: 845-353-4141; e-mail: subscriptions_newyork@cambridge.org; Web site: https://www.cambridge.org/core/what-we-publish/journals
Publication Type: Journal Articles; Information Analyses
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Author Affiliations: N/A