The use of multilevel modeling (MLM) to analyze nested data has grown in popularity over the years in the study of school psychology. However, with the increase in use, several statistical misconceptions about the technique have also proliferated. We discuss some commonly cited myths and golden rules related to the use of MLM, explain their…

A latent variable modeling procedure that can be used to evaluate intraclass correlation coefficients in two-level settings with discrete response variables is discussed. The approach is readily applied when the purpose is to furnish confidence intervals at prespecified confidence levels for these coefficients in setups with binary or ordinal…

This Snapshot highlights key findings from a study that used Bayesian stabilization to improve the reliability (long-term stability) of subgroup proficiency measures that the Pennsylvania Department of Education (PDE) uses to identify schools for Targeted Support and Improvement (TSI) or Additional Targeted Support and Improvement (ATSI). The…

The traditional approach to estimating the consistency of school effects across subject areas and the stability of school effects across time is to fit separate value-added multilevel models to each subject or cohort and to correlate the resulting empirical Bayes predictions. We show that this gives biased correlations and these biases cannot be…

Researchers in group counseling often encounter complex data from individual clients who are members of a group. Clients in the same group may be more similar than clients from different groups and this can lead to violations of statistical assumptions. The complexity of the data also means that predictors and outcomes can be measured at both the…

Discipline-based education researchers have a natural laboratory--classrooms, programs, colleges, and universities. Studies that administer treatments to multiple sections, in multiple years, or at multiple institutions are particularly compelling for two reasons: first, the sample sizes increase, and second, the implementation of the treatments…

Moment-based estimators of the between-study variance are very popular when performing random effects meta-analyses. This type of estimation has many advantages including computational and conceptual simplicity. Furthermore, by using these estimators in large samples, valid meta-analyses can be performed without the assumption that the treatment…

This article proposes a latent variable regression four-level hierarchical model (LVR-HM4) that uses a fully Bayesian approach. Using multisite multiple-cohort longitudinal data, for example, annual assessment scores over grades for students who are nested within cohorts within schools, the LVR-HM4 attempts to simultaneously model two types of…

Item response theory (IRT) models are the main psychometric approach for the development, evaluation, and refinement of multi-item instruments and scaling of latent traits, whereas multilevel models are the primary statistical method when considering the dependence between person responses when primary units (e.g., students) are nested within…

Effective professional learning evaluation requires consideration of five critical stages or levels of information. These five levels, which are presented in this article, represent an adaptation of an evaluation model developed by Kirkpatrick (1959, 1998) for judging the value of supervisory training programs in business and industry.…

Design-based methods have recently been developed as a way to analyze data from impact evaluations of interventions, programs, and policies. The impact estimators are derived using the building blocks of experimental designs with minimal assumptions, and have good statistical properties. The methods apply to randomized controlled trials (RCTs) and…

Research into open education has identified a "high number of participants" and "unpredictable mixed abilities" as factors responsible for the relatively weak presence of language Massive Open Online Courses (MOOCs). This contribution presents a model for open online language courses that aims to bridge this gap. The tangible…

Longitudinal data analysis in education is the study growth over time. A longitudinal study is one in which repeated observations of the same variables are recorded for the same individuals over a period of time. This type of research is known by many names (e.g., time series analysis or repeated measures design), each of which can imply subtle…

This article combines statistical and applied research perspective showing problems that might arise when measurement error in multilevel compositional effects analysis is ignored. This article focuses on data where independent variables are constructed measures. Simulation studies are conducted evaluating methods that could overcome the…

This conference presentation demonstrates a multilevel model for analyzing single case designs. The model is implemented in the Bayesian program WinBUGS. The authors show how it is possible to estimate a d-statistic like the one in Hedges, Pustejovsky and Shadish (2012) in this program. Results are demonstrated on an example.