Since the advent of the twenty-first century, science has experienced a crisis pertaining to the replicability of quantitative research findings, which has become known as the 'replication crisis'. The replication crisis has particularly afflicted research in the behavioural sciences, and psychology in particular. Given the relevance of psychology to education, it is unsurprising that the replication crisis also presents an issue for quantitative educational research, thus potentially compromising its practical usefulness. This article outlines the replication crisis in psychology, and highlights its significance for quantitative educational research. Greater methodological rigour, aimed at addressing methodological deficiencies in the conduct of scientific research, has been suggested as a response to the replication crisis. Following a review of various calls for greater methodological rigour, the current article argues that, whilst methodological deficiencies may be a contributory factor, there is potentially a more fundamental reason for the replication crisis. The case is made that the non-separability of a measured attribute from the measurement instrument, and the irreducible uncertainty in unmeasured attributes, may be the principal reasons for the replication crisis in psychology and education. Implications of such an explanation for the crisis are articulated in relation to the evidence-based policy agenda in education.