Whether you believe you need a methodologist is very much dependent on academic training and your experience working with methodologists in pursuit of research. Methodologists are indispensable at all phases of the research process but perhaps nowhere more than at the beginning, when formulating your research questions and specifying what it is…

A central theme in the field of survey statistics is estimating population-level quantities through data coming from potentially non-representative samples of the population. Multilevel regression and poststratification (MRP), a model-based approach, is gaining traction against the traditional weighted approach for survey estimates. MRP estimates…

When developing a test, it is essential to ensure that the test is free of items with differential item functioning (DIF). DIF occurs when examinees of equal ability, but from different examinee subgroups, have different chances of getting the item correct. According to the multidimensional perspective, DIF occurs because the test measures more…

Differential bundle functioning (DBF) has been proposed to quantify the accumulated amount of differential item functioning (DIF) in an item cluster/bundle (Douglas, Roussos, and Stout). The simultaneous item bias test (SIBTEST, Shealy and Stout) has been used to test for DBF (e.g., Walker, Zhang, and Surber). Research on DBF may have the…

Examinees with item preknowledge tend to obtain inflated test scores that undermine test score validity. With the availability of process data collected in computer-based assessments, the research on detecting item preknowledge has progressed on using both item scores and response times. Item revisit patterns of examinees can also be utilized as…

Joint attention (JA) is the purposeful coordination of an individual's focus of attention with that of another and begins to develop within the first year of life. Delayed, or atypically developing, JA is an early behavioural sign of many developmental disabilities and so assessing JA in infancy can improve our understanding of trajectories of…

Multidimensionality and hierarchical data structure are common in assessment data. These design features, if not accounted for, can threaten the validity of the results and inferences generated from factor analysis, a method frequently employed to assess test dimensionality. In this article, we describe and demonstrate the application of the…

Election forecasting has become the centerpiece of media coverage of elections. Yet for all the attention paid to forecasting, public understanding remains low and increasingly distrustful. We can improve citizen knowledge and comprehension and increase student engagement by giving students the opportunity to develop their own election forecast.…

Text is a burgeoning data source for psychological researchers, but little methodological research has focused on adapting popular modeling approaches for text to the context of psychological research. One popular measurement model for text, topic modeling, uses a latent mixture model to represent topics underlying a body of documents. Recently,…

In cluster randomized evaluations, a treatment or intervention is randomly assigned to a set of clusters each with constituent individual units of observations (e.g., student units that attend schools, which are assigned to treatment). One consideration of these designs is how many units are needed per cluster to achieve adequate statistical…

In cluster randomized evaluations, a treatment or intervention is randomly assigned to a set of clusters each with constituent individual units of observations (e.g., student units that attend schools, which are assigned to treatment). One consideration of these designs is how many units are needed per cluster to achieve adequate statistical…

Assessing sensitivity to unmeasured confounding is an important step in observational studies, which typically estimate effects under the assumption that all confounders are measured. In this paper, we develop a sensitivity analysis framework for balancing weights estimators, an increasingly popular approach that solves an optimization problem to…

Since social networks analysis in education offers valuable insights into social structures and social dynamics that shapes individuals behaviors and information storage and transmission, it has become a hot topic in educational science studies. The aim of this study is to examine the educational sciences studies conducted at higher education…

In studies where the response variable is measured on a ratio scale, a ratio of means or medians provides a standardized measure of effect size that is an alternative to the popular standardized mean difference. Confidence intervals for ratios of population means and medians in independent-samples designs and paired-samples designs are proposed as…

Practitioners of adult education have a long history of teaching for social change. They may, however, be uncomfortable using quantitative methods to assess the impact of their learning activities, or they might lack access to statistical analysis software. Quantitative methods help the practitioner determine whether behavioral or attitudinal…