OrgoPrep, a summer preparatory program integrating multiple active-learning elements (i.e., interactive videos with problem-solving and feedback, synchronous peer-led instruction, and collaborative work), was previously shown to improve academic outcomes in organic chemistry for all students. The present study examined how OrgoPrep differentially…

A simulation study was conducted to investigate the heuristics of the SIBTEST procedure and how it compares with ETS classification guidelines used with the Mantel-Haenszel procedure. Prior heuristics have been used for nearly 25 years, but they are based on a simulation study that was restricted due to computer limitations and that modeled item…

This study offers an approach to testing for differential item functioning (DIF) in a recently developed measurement framework, referred to as "D"-scoring method (DSM). Under the proposed approach, called "P-Z" method of testing for DIF, the item response functions of two groups (reference and focal) are compared by…

To present time-varying evidence, cumulative meta-analysis (CMA) updates results of previous meta-analyses to incorporate new study results. We investigate the properties of CMA, suggest possible improvements and provide the first in-depth simulation study of the use of CMA and CUSUM methods for detection of temporal trends in random-effects…

With the increasing use of automated scores in operational testing settings comes the need to understand the ways in which they can yield biased and unfair results. In this paper, we provide a brief survey of some of the ways in which the predictive methods used in automated scoring can lead to biased, and thus unfair automated scores. After…

This dissertation address two significant challenges in the causal inference workflow for Big Observational Data. The first is designing Big Observational Data with high-dimensional and heterogeneous covariates. The second is performing uncertainty quantification for estimates of causal estimands that are obtained from the application of black box…

Many forms of bias have been identified in higher education research, and in educational and social research in general. This article identifies a further form of bias, positivity bias, and places it in this broader context. Positivity bias is the tendency, in some forms of published higher education research, to only or chiefly report examples of…

Research has examined the influence of a graduate student matching their advisor's demographic characteristics on a variety of outcomes, but comparatively few studies have examined students' preferences concerning such matching. Using data from a national survey of U.S. graduate students in five natural and social science disciplines, the analyses…

This experimental study examined whether the uninformative anchoring effect, which should be ignored, on judgments of learning (JOLs) was eliminated through the learning experience. In the experiments, the participants were asked to predict whether their performance on an upcoming test would be higher or lower than the anchor value (80% in the…

"Attribute-framing bias" reflects people's tendency to evaluate objects framed positively more favorably than the same objects framed negatively. Although biased by the framing valence, evaluations are nevertheless calibrated to the magnitude of the target attribute. In three experiments that manipulated magnitudes in different ways, we…

In a justice-oriented antiracist assessment process, attention to the disruption of white supremacy must occur at every stage--from construct articulation to score reporting. An important step in the assessment development process is the item review stage often referred to as Bias/Fairness and Sensitivity Review. I argue that typical approaches to…

Social desirability bias (SDB) has been a major concern in educational and psychological assessments when measuring latent variables because it has the potential to introduce measurement error and bias in assessments. Person-fit indices can detect bias in the form of misfitted response vectors. The objective of this study was to compare the…

During the first year of life, children begin to develop preferences for their own racial group over others. To interrupt the development of these and other biases during infancy and toddlerhood, educators can use books to promote anti-racist and anti-bias thinking and behaviors in children, while also supporting children's emergent literacy. This…

In this article, the author shares some tips for improving university-wide assessment conversations, focusing on meetings with faculty creating and doing assessment. The lessons are from a multi-year project where the author observed meetings and interviewed faculty about their involvement in university-wide assessment.

Purpose: Judging bias is ironically an inherent risk in every competition, which might threaten the fairness and legitimacy of the competition. The patriotism effect represents one source of judging bias as the judge favors contestants who share the same sentiments, such as the nationalistic, racial, or cultural aspects. This study attempts to…