Replication provides a confrontation of psychological theory, not only in experimental research, but also in model-based research. Goodness-of-fit (GOF) of the original model to the replication data is routinely provided as meaningful evidence of replication. We demonstrate, however, that GOF obscures important differences between the original and…

According to Bayesian models of judgment, testimony from independent informants has more evidential value than dependent testimony. Three experiments investigated learners' sensitivity to this distinction. Each experiment used a social version of the balls-and-urns task, in which participants judged which of two urns was the most likely source of…

Rare events meta-analyses of randomized controlled trials (RCTs) are often underpowered because the outcomes are infrequent. Real-world evidence (RWE) from non-randomized studies may provide valuable complementary evidence about the effects of rare events, and there is growing interest in including such evidence in the decision-making process.…

We investigated the relationship between category learning and domain-general object recognition ability (o). We assessed this relationship in a radiological context, using a category learning test in which participants judged whether white blood cells were cancerous. In study 1, Bayesian evidence negated a relationship between o and category…

Background/Context: Methodological background: Meta-analysis typically depends on the assumption that true effects follow the normal distribution. While assuming normality of effect "estimates" is often supported by a central limit theorem, normality for the distribution of interventions' "true" effects is a computational…

A major problem in human cognition is to understand how newly acquired information and long-standing beliefs about the environment combine to make decisions and plan behaviors. Over-dependence on long-standing beliefs may be a significant source of suboptimal decision-making in unusual circumstances. While the contribution of long-standing beliefs…

Null hypothesis testing has long since been the 'go-to analytic approach' in quantitative second language (L2) research (Norris, 2015, p. 97). To many, however, years of reliance on this approach has resulted in a crisis of inference across the social and behavioral sciences (e.g., Rouder et al., 2016). As an alternative to the null hypothesis…

Bayesian and classical statistical approaches are based on different types of logical principles. In order to avoid mistaken inferences and misguided interpretations, the practitioner must respect the inference rules embedded into each statistical method. Ignoring these principles leads to the paradoxical conclusions that the hypothesis…

Verbal deception detection has gained momentum as a technique to tell truth-tellers from liars. At the same time, researchers' degrees of freedom make it hard to assess the robustness of effects. Replication research can help evaluate how reproducible an effect is. We present the first replication in verbal deception research whereby ferry…

This paper proposes a new method to evaluate informative hypotheses for meta-analysis of Cronbach's coefficient alpha using a Bayesian approach. The coefficient alpha is one of the most widely used reliability indices. In meta-analyses of reliability, researchers typically form specific informative hypotheses beforehand, such as "alpha of…

A key assumption of dual process theory is that reasoning is an explicit, effortful, deliberative process. The present study offers evidence for an implicit, possibly intuitive component of reasoning. Participants were shown sentences embedded in logically valid or invalid arguments. Participants were not asked to reason but instead rated the…

Convergent research suggests that people with ASD have difficulties localizing sounds in space. These difficulties have implications for communication, the development of social behavior, and quality of life. Recently, a theory has emerged which treats perceptual symptoms in ASD as the product of impairments in implicit Bayesian inference; as…

In 1881, Donald MacAlister posed a problem in the "Educational Times" that remains relevant today. The problem centers on the statistical evidence for the effectiveness of a treatment based on a comparison between two proportions. A brief historical sketch is followed by a discussion of two default Bayesian solutions, one based on a…

Inferences about student knowledge, skills, and attributes based on digital activity still largely come from whether students ultimately get a correct result or not. However, the ability to collect activity stream data as individuals interact with digital environments provides information about students' processes as they progress through learning…

This policy brief reviews evidence about the extent to which disadvantaged students are taught by teachers with lower value-added estimates of performance, and seeks to reconcile differences in findings from different studies. We demonstrate that much of the inequity in teacher value added in Washington state is due to differences across different…