The "Stabilizing Subgroup Proficiency Results to Improve the Identification of Low-Performing Schools" study used Bayesian stabilization to improve the reliability (long-term stability) of subgroup proficiency measures that the Pennsylvania Department of Education (PDE) uses to identify schools for Targeted Support and Improvement (TSI)…

Dependent effect sizes are ubiquitous in meta-analysis. Using Monte Carlo simulation, we compared the performance of 2 methods for meta-regression with dependent effect sizes--robust variance estimation (RVE) and 3-level modeling--with the standard meta-analytic method for independent effect sizes. We further compared bias-reduced linearization…

The current study proposes novel methods to predict multistage testing (MST) performance without conducting simulations. This method, called MST test information, is based on analytic derivation of standard errors of ability estimates across theta levels. We compared standard errors derived analytically to the simulation results to demonstrate the…

Item exposure control is, relative to adaptive testing, a nascent concept that has emerged only in the last two to three decades on an academic basis as a practical issue in high-stakes computerized adaptive tests. This study aims to implement a new strategy in item exposure control by incorporating the standard error of the ability estimate into…

We offer a new conceptualization and measurement models for constructs at the group-level of analysis in small group research. The conceptualization starts with classical notions of group behavior proposed by Tönnies, Simmel, and Weber and then draws upon plural subject theory by philosophers Gilbert and Tuomela to frame a new perspective…

Studies analyzing clustered data sets using both multilevel models (MLMs) and ordinary least squares (OLS) regression have generally concluded that resulting point estimates, but not the standard errors, are comparable with each other. However, the accuracy of the estimates of OLS models is important to consider, as several alternative techniques…

Rubrics are widely used in higher education to assess performance in project-based learning environments. To date, the sources of error that may affect their reliability have not been studied in depth. Using generalisability theory as its starting-point, this article analyses the influence of the assessors and the criteria of the rubrics on the…

A compositional effect is when pupil attainment is associated with the characteristics of their peers, over and above their own individual characteristics. Pupils at academically selective schools, for example, tend to out-perform similar-ability pupils who are educated with mixed-ability peers. Previous methodological studies however have shown…

Purpose: Teacher ratings were used to compare children with developmental language disorders (DLD) and their typically developing peers on 2 subtypes of social withdrawal (shyness and unsociability). Measurement invariance analysis was utilized to determine if teachers rated the 2 groups using the same underlying construct for each of the rating…

Latent interaction models have motivated a great deal of methodological research, mainly in the area of estimating such models. Product-indicator methods have been shown to be competitive with other methods of estimation in terms of parameter bias and standard error accuracy, and their continued popularity in empirical studies is due, in part, to…

The field of econometrics largely started with time series analysis because many early datasets were time-series macroeconomic data. As the field developed, more cross-sectional and longitudinal datasets were collected, which today dominate the majority of academic empirical research. In nonacademic (private sector, central bank, and governmental)…

Whenever multiple observers provide ratings, even of the same performance, inter-rater variation is prevalent. The resulting "idiosyncratic rater variance" is considered to be unusable error of measurement in psychometric models and is a threat to the defensibility of our assessments. Prior studies of inter-rater variation in clinical…

Multilevel models (MLMs) have proven themselves to be very useful in social science research, as data from a variety of sources is sampled such that individuals at level-1 are nested within clusters such as schools, hospitals, counseling centers, and business entities at level-2. MLMs using restricted maximum likelihood estimation (REML) provide…

When researchers fit statistical models to multiply imputed datasets, they have to fit the model separately for each imputed dataset. Since there are multiple datasets, there are always multiple sets of model results. It is possible for some of these sets of results not to converge while some do converge. This study examined occurrence of such a…

Non-cognitive competencies play critical role in education. Interventions targeted at these competencies require access to valid measures. Within Indian context, no such validated measure exists at present, and teachers depend on their subjective evaluation of students' non-cognitive competencies represented via grade or comment on report card.…