ERIC Number: EJ1274836
Record Type: Journal
Publication Date: 2020-Nov
Pages: 21
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1759-2879
EISSN: N/A
Available Date: N/A
A Generalized-Weights Solution to Sample Overlap in Meta-Analysis
Research Synthesis Methods, v11 n6 p812-832 Nov 2020
Meta-studies are often conducted on empirical findings obtained from overlapping samples. Sample overlap is common in research fields that strongly rely on aggregated observational data (eg, economics and finance), where the same set of data may be used in several studies. More generally, sample overlap tends to occur whenever multiple estimates are sampled from the same study. We show analytically how failing to account for sample overlap causes high rates of false positives, especially for large meta-sample sizes. We propose a generalized-weights (GW) meta-estimator, which solves the sample overlap problem by explicitly modeling the variance-covariance matrix that describes the structure of dependence among estimates. We show how this matrix can be constructed from information that is usually available from basic sample descriptions in the primary studies (ie, sample sizes and number of overlapping observations). The GW meta-estimator amounts to weighting each empirical outcome according to its share of independent sampling information. We use Monte Carlo simulations to (a) demonstrate how the GW meta-estimator brings the rate of false positives to its nominal level, and (b) quantify the efficiency gains of the GW meta-estimator relative to standard meta-estimators. The GW meta-estimator is fairly straightforward to implement and can solve any case of sample overlap, within or between studies.
Descriptors: Meta Analysis, Sampling, Research Problems, Computation, Statistical Analysis, Correlation
Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www-wiley-com.bibliotheek.ehb.be/en-us
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Data File: URL: http://doi.org.bibliotheek.ehb.be/10.17605/OSF.IO/KS4NT
Author Affiliations: N/A