Evidence-based education aims to support policy makers choosing between potential interventions. This rarely involves considering each in isolation; instead, sets of evidence regarding many potential policy interventions are considered. Filtering a set on any quantity measured with error risks the "winner's curse": conditional on…

Large-scale analyses are a powerful and increasingly common tool for investigating a range of public health and social concerns (Pienta, O'Rourke, & Franks, 2011). This series will provide a primer on large-scale secondary analysis in school psychology, with this article focusing on considerations for researchers interested in applying and…

The article positions mixed-method research (MMR) as a principled complementary research method to the traditional quantitative and qualitative research approaches. By situating MMR in an analysis of some of the common research paradigms, the article presents it as a natural choice in order to complement and cater to the increasingly complex needs…

The article positions mixed-method research (MMR) as a principled complementary research method to the traditional quantitative and qualitative research approaches. By situating MMR in an analysis of some of the common research paradigms, the article presents it as a natural choice in order to complement and cater to the increasingly complex needs…

Social science datasets usually have missing cases, and missing values. All such missing data has the potential to bias future research findings. However, many research reports ignore the issue of missing data, only consider some aspects of it, or do not report how it is handled. This paper rehearses the damage caused by missing data. The paper…

It is widely believed that regression models for binary responses are problematic if we want to compare estimated coefficients from models for different groups or with different explanatory variables. This concern has two forms. The first arises if the binary model is treated as an estimate of a model for an unobserved continuous response and the…

Quantitative social scientists have adopted the positivist epistemology and methodology of natural sciences, seeking objectivity, generalisability, and neutrality. However, in social sciences -- unlike in natural sciences -- humans are both the investigators and the object of investigation, leading to intricate interconnections between researchers…

Although the null hypothesis significance testing procedure is problematic, many still favor the use of "p"-values as indicating the state of evidence against the model used to generate the "p"-value. From this perspective, "p"-values benefit science; or would benefit science if used correctly. In contrast, the novel…

Panel data methods, which include difference-in-differences and comparative interrupted time series, have become increasingly common in education policy research. The key idea is to use variation across time and space (e.g., school districts) to estimate the effects of policy or programmatic changes that happen in some localities but not others.…

Panel data methods, which include difference-in-differences and comparative interrupted time series, have become increasingly com- mon in education policy research. The key idea is to use variation across time and space (e.g., school districts) to estimate the effects of policy or programmatic changes that happen in some localities but not others.…

Contrary to its typical presentation in scientific publications as a certain and linear process, in reality, the experimental method, not least the design aspect of it, requires a great deal of trial-and-error and ad hoc decision-making on the part of the researchers. This uncertain and contingent aspect of research, although little known outside…

For binary experimental data, we discuss randomization-based inferential procedures that do not need to invoke any modeling assumptions. We also introduce methods for likelihood and Bayesian inference based solely on the physical randomization without any hypothetical super population assumptions about the potential outcomes. These estimators have…

In systematic reviews, meta-analyses are routinely applied to summarize the results of the relevant studies for a specific research question. If one can assume that in all studies the same true effect is estimated, the application of a meta-analysis with common effect (commonly referred to as fixed-effect meta-analysis) is adequate. If…

Researchers are often interested in establishing equivalence of population variances. Traditional difference-based procedures are appropriate to answer questions about differences in some statistic (e.g., variances, etc.). However, if a researcher is interested in evaluating the equivalence of population variances, it is more appropriate to use a…

We present an aggregation scheme that increases power in randomized controlled trials and quasi-experiments when the intervention possesses a robust and well-articulated theory of change. Intervention studies using longitudinal data often include multiple observations on individuals, some of which may be more likely to manifest a treatment effect…