This case study explores the use of large language models (LLMs) as analytical partners for data exploration and interpretation. Grounded in original research, it navigates the intricacies of using LLMs for uncovering themes from datasets. The study tackles various methodological and practical challenges encountered during the research process…

The U.S. Department of Education's Office of Special Education Programs collects data on how states implement the Individuals with Disabilities Education Act through the mandated State Performance Plan/Annual Performance Report (SPP/APR). Some indicators in the SPP/APR require state educational agencies (SEAs) to report data by race and ethnicity.…

Background: Data missingness can bias interpretation and outcomes resulting from data use. We describe data missingness in the longest-standing US-based youth fitness surveillance system (2006/07-2019/20). Methods: This observational study uses the New York City FITNESSGRAM (NYCFG) database from 1,983,629 unique 4th-12th grade students (9,147,873…

We used multitrait-multimethod (MTMM) modeling to examine general factors of psychopathology in three samples of youth (Ns = 2119, 303, 592) for whom three informants reported on the youth's psychopathology (e.g., child, parent, teacher). Empirical support for the "p"-factor diminished in multi-informant models compared with…

Omitted variable bias occurs in most statistical models. Whenever a confounding variable that is correlated with both dependent and independent variables is omitted from a statistical model, estimated effects of included variables are likely to be biased due to omitted variables. This issue is particularly problematic in physics education research…

Several programs of research have sought to assess the replicability of scientific findings in different fields, including economics and psychology. These programs attempt to replicate several findings and use the results to say something about large-scale patterns of replicability in a field. However, little work has been done to understand the…

Basic knowledge of ideas of causal inference can help students to think beyond data, that is, to think more clearly about the data generating process. Especially for (maybe big) observational data, qualitative assumptions are important for the conclusions drawn and interpretation of the quantitative results. Concepts of causal inference can also…

The popularity of online surveys has increased the prominence of using weights that capture units' probabilities of inclusion for claims of representativeness. Yet, much uncertainty remains regarding how these weights should be employed in analysis of survey experiments: Should they be used or ignored? If they are used, which estimators are…

This article engages the philosophy of science of data, with a focus on the extent to which data are always already imbued with racializations. As Alexander Weheliye has argued, racializations are not to be reduced to race but rather is the sociopolitical process that hierarchizes and differentiates bodies producing the entangled by-products of…

Education researchers often study count variables, such as times a student reached a goal, discipline referrals, and absences. Most researchers that study these variables use typical regression methods (i.e., ordinary least-squares) either with or without transforming the count variables. In either case, using typical regression for count data can…

Heterogeneity in response styles can affect the conclusions drawn from rating scale data. In particular, biased estimates can be expected if one ignores a tendency to middle categories or to extreme categories. An adjacent categories model is proposed that simultaneously models the content-related effects and the heterogeneity in response styles.…

Research has shown that forest plots are a gold standard in the visualization of meta-analytic results. However, research on the general interpretation of forest plots and the role of researchers' meta-analysis experience and field of study is still unavailable. Additionally, the traditional display of effect sizes, confidence intervals, and…

This paper examines sources of potential bias in systematic reviews and meta-analyses which can distort their findings, leading to problems with interpretation and application by practitioners and policymakers. It follows from an article that was published in the "Canadian Journal of Communication" in 1990, "Integrating Research…

A major obstacle to developing evidenced-based policy is the difficulty of implementing randomized experiments to answer all causal questions of interest. When using a nonexperimental study, it is critical to assess how much the results could be affected by unmeasured confounding. We present a set of graphical and numeric tools to explore the…

This dissertation comprises three papers that propose, discuss, and illustrate models to make improved inferences about research questions regarding student achievement in education. Addressing the types of questions common in educational research today requires three different "extensions" to traditional educational assessment: (1)…