Evaluators of education interventions are increasingly designing studies to detect impacts much smaller than the 0.20 standard deviations that Cohen (1988) characterized as "small." While the need to detect smaller impacts is based on compelling arguments that such impacts are substantively meaningful, the drive to detect smaller impacts…

Purpose: Studies in the literature have generally demonstrated that the causes of differential item functioning (DIF) are complex and not directly related to defined groups. The purpose of this study is to determine the DIF according to the mixture item response theory (MixIRT) model, based on the latent group approach, as well as the…

As Social and Emotional Learning (SEL) expands to focus on adolescent populations, the broadly accepted theoretical framework put forth by the Collaborative for Academic, Social, and Emotional Learning (CASEL) should be empirically tested for measurement utility. Using longitudinal data from the 4-H Study of Positive Youth Development, we first…

As educational practitioners and policymakers expand the range of student outcomes they assess, student perception surveys--particularly those targeting social-emotional learning--have grown in popularity. Despite excitement around the potential for measuring a wider array of important student outcomes, concerns about the validity of the…

Measures of school-level growth in student outcomes are common tools used to assess the impacts of schools. The vast majority of these measures are based on standardized tests, even though emerging evidence demonstrates the importance of social-emotional skills (SEL). This paper uses the first large-scale panel surveys of students on SEL to…

The purpose of this inquiry was to investigate the effectiveness of item response theory (IRT) proficiency estimators in terms of estimation bias and error under multistage testing (MST). We chose a 2-stage MST design in which 1 adaptation to the examinees' ability levels takes place. It includes 4 modules (1 at Stage 1, 3 at Stage 2) and 3 paths…

The authors respond to Goodwin's (2014) "Threat Assessment: Are We Using the Wrong Nets?" and use that article as a springboard to discuss various aspects of the threat assessment and management process. We find that Goodwin misses critical elements of the threat assessment and management process, conflates the process with two types of…

We examine the factor structure of scores from the CLASS-S protocol obtained from observations of middle school classroom teaching. Factor analysis has been used to support both interpretations of scores from classroom observation protocols, like CLASS-S, and the theories about teaching that underlie them. However, classroom observations contain…

The method of least-squares (LS) has a built-in procedure for estimating the standard errors (SEs) of the adjustable parameters in the fit model: They are the square roots of the diagonal elements of the covariance matrix. This means that one can use least-squares to obtain numerical values of propagated errors by defining the target quantities as…

This article examines the relationship between preschool quality and children's early development in a sample of over 7900 children enrolled in 578 preschools in rural Indonesia. Quality was measured by: (1) classroom observations using the Early Childhood Environment Rating Scale-Revised (ECERS-R); (2) teacher characteristics; and (3) structural…

Measurement is an important component of K-grade 12 mathematics curricula. The concepts of area and perimeter of polygons are first introduced in third grade and serve as the basis for teaching in the upper grades. Without a strong understanding of measurement, students will struggle to meaningfully grasp three-dimensional measurement concepts…

Formative assessment is a classroom practice that has received much attention in recent years for its established potential at increasing student learning. A frequent analytic approach for determining the quality of formative assessment practices is to develop a coding scheme and determine frequencies with which the codes are observed; however,…

This study illustrates an integrative psychometric framework to investigate two sources of construct-relevant multidimensionality in answers to the Self-Perception Profile for Children (SPPC). Using a sample of 2,353 German students attending Grades 3 to 6, we contrasted: (a) first-order versus hierarchical and bifactor models to investigate…

The design of research studies utilizing binary multilevel models must necessarily incorporate knowledge of multiple factors, including estimation method, variance component size, or number of predictors, in addition to sample sizes. This Monte Carlo study examined the performance of random effect binary outcome multilevel models under varying…

Program evaluation often suffers due to time constraints, imperfect instruments, incomplete data, and the need to report standardized metrics. This article about the evaluation process for the Wi$eUp financial education program showcases the difficulties inherent in evaluation and suggests best practices for assessing program effectiveness. We…