In multisite trials, learning about treatment effect variation across sites is critical for understanding where and for whom a program works. Unadjusted comparisons, however, capture "compositional" differences in the distributions of unit-level features as well as "contextual" differences in site-level features, including…

This article offers some less-than-rigorous explanations for the notion of degrees of freedom, and for the particular formulae to be used when computing those values.

Recently, quantitative researchers have shown increased interest in two-step factor score regression (FSR) approaches to structural model estimation. A particularly promising approach proposed by Croon involves first extracting factor scores for each latent factor in a larger model, then correcting the variance-covariance matrix of the factor…

In multisite trials, learning about treatment effect variation across sites is critical for understanding where and for whom a program works. Unadjusted comparisons, however, capture "compositional" differences in the distributions of unit-level features as well as "contextual" differences in site-level features, including…

A visual aid for teaching the formula for the Mann-Whitney U statistic is proposed, inspired by 'proof without words', a didactic exercise from the mathematics literature that uses a visual approach to justify mathematical statements, including formulas and theorems.

This article advocates that introductory statistics be taught by basing all calculations on a single simple margin-of-error formula and deriving all of the standard introductory statistical concepts (confidence intervals, significance tests, comparisons of means and proportions, etc) from that one formula. It is argued that this approach will…

The 0.70 and 0.80 reliability standards, proposed by Jum Nunnally, are widely employed across a spectrum of research domains. Nonetheless, due to their arbitrary nature, both standards fail to satisfy the needs of researchers. This paper presents a set of formulas that can be used to compute minimum reliability standards as a function of a…

Autocorrelation and partial autocorrelation, which provide a mathematical tool to understand repeating patterns in time series data, are often used to facilitate the identification of model orders of time series models (e.g., moving average and autoregressive models). Asymptotic methods for testing autocorrelation and partial autocorrelation such…

A relevant aim of research in education is to find and study the reasoning lines that students deploy when dealing with problematic situations. This can be done through an analysis of the answers students give to a questionnaire. In this paper, we discuss some methodological aspects involved in the quantitative analysis of a questionnaire by means…

Drawing parallels to classical test theory, this article clarifies the difference between rater accuracy and reliability and demonstrates how category marginal frequencies affect rater agreement and Cohen's kappa. Category assignment paradigms are developed: comparing raters to a standard (index) versus comparing two raters to one another…

Test score distributions of schools or demographic groups are often summarized by frequencies of students scoring in a small number of ordered proficiency categories. We show that heteroskedastic ordered probit (HETOP) models can be used to estimate means and standard deviations of multiple groups' test score distributions from such data. Because…

In causal mediation analysis, the definitions of the natural direct and indirect effects involve potential outcomes that can never be observed, so-called a priori counterfactuals. This conceptual challenge translates into issues in identification, which requires strong and often unverifiable assumptions, including sequential ignorability.…

In education, sample characteristics can be complex due to the nested structure of students, teachers, classrooms, schools, and districts. In the past, not many considerations were given to such complex sampling schemes in statistical power analysis. More recently in the past two decades, however, education scholars have developed tools to conduct…

Like studies focused on the detection of a main or total effect, a principal consideration in the design of studies examining mediation is the power with which mediation effects can be detected if they exist (e.g., Raudenbush, 1997). However, unlike studies concerning the detection of main effects, literature has not established power formulas for…

In investigations of unusual testing behavior, a common question is whether a specific pattern of responses occurs unusually often within a group of examinees. In many current tests, modern communication techniques can permit quite large numbers of examinees to share keys, or common response patterns, to the entire test. To address this issue,…