This article presents two alternative methods to null hypothesis significance testing (NHST) for improving inferences from underpowered research designs. Post hoc design analysis (PHDA) assesses whether an NHST analysis generating null findings might otherwise have had sufficient power to detect effects of plausible magnitudes. Bayesian analysis…

Observations are widely used in research and evaluation to characterize teaching and learning activities. Because conducting observations is typically resource intensive, it is important that inferences from observation data are made confidently. While attention focuses on interrater reliability, the reliability of a single-class measure over the…

Periodic changes in occupational classifications make it difficult to obtain consistent measures of social class over time, potentially jeopardizing research on class-based trends. The severity of this problem depends, in part, on the measurement strategies used to address those changes. The authors propose that when a sample has been coded partly…

This research seeks to understand how communicators' strategy towards determining if an argument would be perceived as persuasive by someone else may be affected by their social judgements of that person. In so doing, this research contributes to our larger understanding of how social categorization processes affect communication by applying a…

The synthetic control method (SCM) is a popular approach for estimating the impact of a treatment on a single unit in panel data settings. The "synthetic control" is a weighted average of control units that balances the treated unit's pre-treatment outcomes and other covariates as closely as possible. A critical feature of the original…

This study was conducted to better understand the difficulties faced by underprepared college readers, including those enrolled in Developmental Education (DE) programs. The Reading Systems Framework was used as a theoretical motivation. Participants (n = 258) completed a measure of component reading skills (Study Aid and Reading Assessment, word…

Randomized A/B tests within online learning platforms enable us to draw unbiased causal estimators. However, precise estimates of treatment effects can be challenging due to minimal participation, resulting in underpowered A/B tests. Recent advancements indicate that leveraging auxiliary information from detailed logs and employing design-based…

Researchers interested in studying the frequency of events or behaviors among a population must rely on count data provided by sampled individuals. Often, this involves a decision between live event counting, such as a behavioral diary, and recalled aggregate counts. Diaries are generally more accurate, but their greater cost and respondent burden…

Conventional advice discourages controlling for postoutcome variables in regression analysis. By contrast, we show that controlling for commonly available postoutcome (i.e., future) values of the treatment variable can help detect, reduce, and even remove omitted variable bias (unobserved confounding). The premise is that the same unobserved…

Courses: Research methods for undergraduates or graduates. Objectives: The aims of this activity are: (1) to clarify the basics of experimental design; (2) to illustrate the concept of levels of measurement; (3) to demonstrate in-person/hands-on data collection procedures; (4) to understand and practice the steps in null hypothesis testing; and…

This study examined the direct and indirect relations of foundational language skills (vocabulary, syntactic knowledge, and orthographic knowledge), higher-order cognitive skills (inference making and comprehension monitoring), and word reading to reading comprehension in Chinese. Consistent with the hierarchical relations specified in the Direct…

Technical documentation for educational tests focuses primarily on properties of individual scores at single points in time. Reliability, standard errors of measurement, item parameter estimates, fit statistics, and linking constants are standard technical features that external stakeholders use to evaluate items and individual scale scores.…

Analogical reasoning is a core facet of higher cognition in humans. Creating analogies as we navigate the environment helps us learn. Analogies involve reframing novel encounters using knowledge of familiar, relationally similar contexts stored in memory. When an analogy links a novel encounter with a familiar context, it can aid in problem…

Proving and refuting are fundamental aspects of mathematical practice that are intertwined in mathematical activity in which conjectures and proofs are often produced and improved through the back-and-forth transition between attempts to prove and disprove. One aspect underexplored in the education literature is the connection between this…

The purpose of this paper is to highlight issues related to students' personal inferences that arise when students verbally explain their justification for calculus statements. We conducted clinical interviews with three undergraduate students who had taken first-semester calculus but had not yet been exposed to formal proof writing activities…