Large language models (LLMs) provide cost-effective but possibly inaccurate predictions of human behavior. Despite growing evidence that predicted and observed behavior are often not "interchangeable," there is limited guidance on using LLMs to obtain valid estimates of causal effects and other parameters. We argue that LLM predictions…

In education settings, treatments are often non-randomly assigned to clusters, such as schools or classrooms, while outcomes are measured for students. This research design is called the clustered observational study (COS). We examine the consequences of common support violations in the COS context. Common support violations occur when the…

Joint attention (JA) is the purposeful coordination of an individual's focus of attention with that of another and begins to develop within the first year of life. Delayed, or atypically developing, JA is an early behavioural sign of many developmental disabilities and so assessing JA in infancy can improve our understanding of trajectories of…

Quantitative reasoning (QR) is a key skill for undergraduate biology education. Despite this, many students struggle with QR. Here, we use the theoretical framework of student noticing to investigate why some students struggle with QR in introductory biology labs. Under this framework, what students notice when given new information and data…

Educational reform priorities such as emphasis on quantitative modelling (QM) have positioned undergraduate biology instructors as designers of QM experiences to engage students in authentic science practices that support the development of data-driven and evidence-based reasoning. Yet, little is known about how biology instructors adapt to the…

The vast majority of research involving active learning pedagogies uses passive lecture methods as a baseline. We propose to move beyond such comparisons to understand the mechanisms that make different active learning styles unique. Here, we use COPUS observations to record student and instructor activities in six known styles of active learning…

This paper aims to bring into the ethical debate on covert research two aspects that are neglected to date: the perspective of the research subjects and the special responsibility of investigators towards their observers. Both aspects are falling behind, especially in quantitative social research. From a methodological point of view, quantitative…

Scholars of civil society organizations (CSOs) have long been concerned with face-to-face interactions among participants at meetings, events, and activities--what we collectively call "convenings." Small-N ethnographies have uncovered substantial differences in the dynamics of convenings within and across CSOs. Large-N quantitative…

Clustered observational studies (COSs) are a critical analytic tool for educational effectiveness research. We present a design framework for the development and critique of COSs. The framework is built on the counterfactual model for causal inference and promotes the concept of designing COSs that emulate the targeted randomized trial that would…

Clustered observational studies (COSs) are a critical analytic tool for educational effectiveness research. We present a design framework for the development and critique of COSs. The framework is built on the counterfactual model for causal inference and promotes the concept of designing COSs that emulate the targeted randomized trial that would…

Various tests to check the homogeneity of variance assumption have been proposed in the literature, yet there is no consensus as to their robustness when the assumption of normality does not hold. This simulation study evaluated the performance of 14 tests for the homogeneity of variance assumption in one-way ANOVA models in terms of Type I error…

Two studies investigated the accuracy of eight different interval-based group observation methods that are commonly used to assess the effects of classwide interventions. In Study 1, a Microsoft Visual Basic program was created to simulate a large set of observational data. Binary data were randomly generated at the student level to represent…

We investigated the premise that observation measures of instruction are indicators of effective teaching, using the definition of effectiveness articulated by departments of education: teaching that boosts student achievement. We argued that student motivation is equally as important as achievement in the evaluation of teaching effectiveness…

The purpose of the study was to evaluate bidirectional associations between peer acceptance and both emotion and effortful control during kindergarten (N = 301). In both the fall and spring semesters, we obtained peer nominations of acceptance, measures of positive and negative emotion based on naturalistic observations in school (i.e., classroom,…

This study aims to evaluate critical thinking skills in a multidimensional way. Based on this objective, the level of teachers' critical thinking skills, how they conceptualize critical thinking, their practices of critical thinking in the classroom, and if the critical thinking is referred enough in the curriculum are analyzed in this research.…