Combining observational and experimental data for causal inference can improve treatment effect estimation. However, many observational data sets cannot be released due to data privacy considerations, so one researcher may not have access to both experimental and observational data. Nonetheless, a small amount of risk of disclosing sensitive…

Mediation analysis is widely applied in various fields of science, such as psychology, epidemiology, and sociology. In practice, many psychological and behavioral phenomena are dynamic, and the corresponding mediation effects are expected to change over time. However, most existing mediation methods assume a static mediation effect over time,…

Item-level fit analysis not only serves as a complementary check to global fit analysis, it is also essential in scale development because the fit results will guide item revision and/or deletion (Liu & Maydeu-Olivares, 2014). During data collection, missing response data may likely happen due to various reasons. Chi-square-based item fit…

Developing models and using mathematics are two key practices in internationally recognized science education standards such as the Next Generation Science Standards (NGSS, 2013). In this paper, we used a virtual performance-based formative assessment to capture students' competencies at both "developing" and "evaluating"…

Prisms VR, a secondary math learning application, allows for users to see, manipulate, and engage with mathematical concepts in an embodied way in Virtual Reality (VR) environment. We examine cases in which mathematics teachers and middle school students worked through Prisms and reflected upon their experiences. Findings indicate that VR…

Model selection is important in any statistical analysis, and the primary goal is to find the preferred (or most parsimonious) model, based on certain criteria, from a set of candidate models given data. Several recent publications have employed the deviance information criterion (DIC) to do model selection among different forms of multilevel item…

Among current state-of-art estimation methods for multilevel IRT models, the two-stage divide-and-conquer strategy has practical advantages, such as clearer definition of factors, convenience for secondary data analysis, convenience for model calibration and fit evaluation, and avoidance of improper solutions. However, various studies have shown…

Causal inference regarding a hypothesized mediation mechanism relies on the assumptions that there are no omitted pretreatment confounders (i.e., confounders preceding the treatment) of the treatment-mediator, treatment-outcome, and mediator-outcome relationships, and there are no posttreatment confounders (i.e., confounders affected by the…

This paper serves as a practical guide to mediation design and analysis by evaluating the ability of mediation models to detect a significant mediation effect using limited data. The cross-sectional mediation model, which has been shown to be biased when the mediation is happening over time, is compared to longitudinal mediation models:…

Previous research has shown that providing video recommendations to students in virtual learning environments implemented at scale positively affects student achievement. However, it is also critical to evaluate whether the treatment effects are heterogeneous, and whether they depend on contextual variables such as disadvantaged student status and…

A common approach for introducing students to a new science concept is to present them with multiple cases of the phenomenon and ask them to explore. The expectation is that students will naturally take advantage of the multiple cases to support their learning and seek an underlying principle for the phenomenon. However, the success of such tasks…

Causal inference in observational studies typically requires making comparisons between groups that are dissimilar. For instance, researchers investigating the role of a prolonged duration of breastfeeding on child outcomes may be forced to make comparisons between women with substantially different characteristics on average. In the extreme there…

Students conducted inquiry using simulations within a rich learning environment for 4 science topics. By applying educational data mining to students' log data, assessment metrics were generated for two key inquiry skills, testing stated hypotheses and designing controlled experiments. Three models were then developed to analyze the transfer of…