NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ885768
Record Type: Journal
Publication Date: 2010-Jun
Pages: 10
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1082-989X
EISSN: N/A
Available Date: N/A
A Model-Averaging Approach to Replication : The Case of "p[subscript rep]"
Iverson, Geoffrey J.; Wagenmakers, Eric-Jan; Lee, Michael D.
Psychological Methods, v15 n2 p172-181 Jun 2010
The purpose of the recently proposed "p[subscript rep]" statistic is to estimate the probability of concurrence, that is, the probability that a replicate experiment yields an effect of the same sign (Killeen, 2005a). The influential journal "Psychological Science" endorses "p[subscript rep]" and recommends its use over that of traditional methods. Here we show that "p[subscript rep]" overestimates the probability of concurrence. This is because "p[subscript rep]" was derived under the assumption that all effect sizes in the population are equally likely a priori. In many situations, however, it is advisable also to entertain a null hypothesis of no or approximately no effect. We show how the posterior probability of the null hypothesis is sensitive to a priori considerations and to the evidence provided by the data; and the higher the posterior probability of the null hypothesis, the smaller the probability of concurrence. When the null hypothesis and the alternative hypothesis are equally likely a priori, "p[subscript rep]" may overestimate the probability of concurrence by 30% and more. We conclude that "p[subscript rep]" provides an upper bound on the probability of concurrence, a bound that brings with it the danger of having researchers believe that their experimental effects are much more reliable than they actually are. (Contains 4 footnotes and 2 figures.)
American Psychological Association. Journals Department, 750 First Street NE, Washington, DC 20002-4242. Tel: 800-374-2721; Tel: 202-336-5510; Fax: 202-336-5502; e-mail: order@apa.org; Web site: http://www.apa.org/publications
Publication Type: Journal Articles; Reports - Descriptive
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Author Affiliations: N/A