It is common for social scientists to use formal quantitative methods to compare ecological units such as towns, schools, or nations. In many cases, the size of these units in terms of the number of individuals subsumed in each differs substantially. When the variables in question are counts, there is generally some attempt to neutralize…

We provide a novel characterization of augmented balancing weights, also known as automatic debiased machine learning (AutoDML). These popular "doubly robust" or "de-biased machine learning estimators" combine outcome modeling with balancing weights -- weights that achieve covariate balance directly in lieu of estimating and…

Asymmetric Likert-type items in research studies can present several challenges in data analysis, particularly concerning missing data. These items are often characterized by a skewed scaling, where either there is no neutral response option or an unequal number of possible positive and negative responses. The use of conventional techniques, such…

In the field of disparities research, there has been growing interest in developing a counterfactual-based decomposition analysis to identify underlying mediating mechanisms that help reduce disparities in populations. Despite rapid development in the area, most prior studies have been limited to regression-based methods, undermining the…

Missing covariates is a common issue when fitting meta-regression models. Standard practice for handling missing covariates tends to involve one of two approaches. In a complete-case analysis, effect sizes for which relevant covariates are missing are omitted from model estimation. Alternatively, researchers have employed the so-called…

Teachers need instructionally useful data to make timely and appropriate decisions to meet their students with intensive needs (Filderman et al., 2019). Teachers have still experienced difficulty in instructional decision making in response to students' CBM data (Gesel et al., 2021). This is because data itself that was used for simply determining…

In education, machine learning (ML), especially deep learning (DL) in recent years, has been extensively used to improve both teaching and learning. Despite the rapid advancement of ML and its application in education, a few challenges remain to be addressed. In this thesis, in particular, we focus on two such challenges: (i) data scarcity and…

Standard meta-analysis methods are vulnerable to bias from incomplete reporting of results (both publication and outcome reporting bias) and poor study quality. Several alternative methods have been proposed as being less vulnerable to such biases. To evaluate these claims independently we simulated study results under a broad range of conditions…

Meta-analysis is a statistical method that combines quantitative findings from previous studies. It has been increasingly used to obtain more credible results in a wide range of scientific fields. Combining the results of relevant studies allows researchers to leverage study similarities while modeling potential sources of between-study…

Doubly robust (DR) estimators that combine regression adjustments and inverse probability weighting (IPW) are widely used in causal inference with observational data because they are claimed to be consistent when either the outcome or the treatment selection model is correctly specified (Scharfstein et al., 1999). This property of "double…

Composite scores are an exceptionally important psychometric tool for behavioral science research applications. A prototypical example occurs with self-report data, where researchers routinely use questionnaires with multiple items that tap into different features of a target construct. Item-level missing data are endemic to composite score…

The standard regression technique for modeling binary response variables in education research is logistic regression. The odds ratios from these models are used to quantify and communicate variable effects. These effects are sometimes pooled together as in a meta-analysis. We argue that this process is problematic as odds ratios calculated from…

In the complex world of educational policy, causal inferences will be debated. As we review non-experimental designs in educational policy, we focus on how to clarify and focus the terms of debate. We begin by presenting the potential outcomes/counterfactual framework and then describe approximations to the counterfactual generated from the…

It is widely believed that regression models for binary responses are problematic if we want to compare estimated coefficients from models for different groups or with different explanatory variables. This concern has two forms. The first arises if the binary model is treated as an estimate of a model for an unobserved continuous response and the…

Social network data often contain missing values because of the sensitive nature of the information collected and the dependency among the network actors. As a response, network imputation methods including simple ones constructed from network structural characteristics and more complicated model-based ones have been developed. Although past…