Rankings of international higher education institutions have become influential, newsworthy metrics worldwide. This is unsurprising given the rising prevalence of cross-national college attendance. This report evaluates the methodologies of three major international ranking systems: the Academic Ranking of World Universities (ARWU, or…

Conceptual replications have received increased coverage in the educational research agenda. This article argues for clarity in, and justification of, the definition, scope, and boundaries of a conceptual replication and what it can and cannot do. It argues for clear justifications when changing components from those of the original study. The…

Discrimination is associated with numerous psychological health outcomes over the life course. The nine-item Everyday Discrimination Scale (EDS) is one of the most widely used measures of discrimination; however, this nine-item measure may not be feasible in large-scale population health surveys where a shortened discrimination measure would be…

Background: For decades, education researchers have relied on the work of Campbell, Cook, and Shadish to help guide their thinking about valid impact estimates in the social sciences (Campbell & Stanley, 1963; Shadish et al., 2002). The foundation of this work is the "validity typology" and its associated "threats to…

An increased focus on transparency and replication in science has stimulated reform in research practices and dissemination. As a result, the research culture is changing: the use of preregistration is on the rise, access to data and materials is increasing, and large-scale replication studies are more common. In this article, I discuss two…

This dissertation is composed of projects on three aspects of gathering and learning from data in the social sciences: drawing representative samples, taking valid measurement, and making warranted inductive inferences. Chapter one studies the challenge of drawing representative human samples. It is well documented that most samples used by…

Apart from providing a bird's eye view of a given domain of inquiry (Ioannidis et al., 2015), systematic reviews such as methodological syntheses offer a magnifying glass on prevalent existing problems in primary research and furnish empirically grounded solutions to rectify these issues (Page et al., 2021). As survey research in second language…

Generative artificial intelligence (GenAI) has profoundly reshaped how graduate students conceptualise, design and conduct academic research in higher education. While current metrics predominantly focus on technology acceptance and usage patterns, they often neglect the diverse cognitive engagements in graduate students' integration of GenAI…

The purpose of this study is to use Indigenous data collection to present construct validity of an instrument designed to test the American Indian/Alaska Native Millennium Falcon Postsecondary Persistence Model (Lopez, 2018). In the following, I describe an alternative sampling technique based on an Indigenous quantitative methodology to examine…

We offer a new conceptualization and measurement models for constructs at the group-level of analysis in small group research. The conceptualization starts with classical notions of group behavior proposed by Tönnies, Simmel, and Weber and then draws upon plural subject theory by philosophers Gilbert and Tuomela to frame a new perspective…