Analysis of the differential treatment effects across targeted subgroups and contexts is a critical objective in many evaluations because it delineates for whom and under what conditions particular programs, therapies or treatments are effective. Unfortunately, it is unclear how to plan efficient and effective evaluations that include these…

The testlet comprises a set of items based on a common stimulus. When the testlet is used in the tests, there may violate the local independence assumption, and in this case, it would not be appropriate to use traditional item response theory models in the tests in which the testlet is included. When the testlet is discussed, one of the most…

Although Cattaneo et al. (2019) provided a data-driven framework for power computations for Regression Discontinuity Designs in line with rdrobust Stata and R commands, which allows higher-order functional forms for the score variable when using the non-parametric local polynomial estimation, analogous advancements in their parametric estimation…

The Peto odds ratio is a well-known effect measure in meta-analysis of binary outcomes. For pairwise comparisons, the Peto odds ratio estimator can be severely biased in the situation of unbalanced sample sizes in the two treatment groups or large treatment effects. In this publication, we evaluate Peto odds ratio estimators in the setting of…

This paper focuses on the considerations in determining the number of participants for qualitative research because of the lack of clear guidelines in this area. The study has employed a semi-systematic literature review that is embedded with the researcher's experience. The study has concluded that the purpose of the research, methodological…

Moderated multiple regression (MMR) has become a fundamental tool for applied researchers, since many effects are expected to vary based on other variables. However, the inherent complexity of MMR creates formidable challenges for adequately performing power analysis on interaction effects to ensure reliable and replicable research results. Prior…

Peer tutoring has become a widely used practice in higher education institutions to support students' academic success, although its effects remain controversial. This article synthesizes 27 independent experimental and quasi-experimental studies to examine the relationship between peer tutoring programs and college students' academic performance,…

Confirmatory factor analyses (CFA) are often used in psychological research when developing measurement models for psychological constructs. Evaluating CFA model fit can be quite challenging, as tests for exact model fit may focus on negligible deviances, while fit indices cannot be interpreted absolutely without specifying thresholds or cutoffs.…

Purpose: Many studies using machine learning (ML) in speech, language, and hearing sciences rely upon cross-validations with single data splitting. This study's first purpose is to provide quantitative evidence that would incentivize researchers to instead use the more robust data splitting method of nested k-fold cross-validation. The second…

Despite recent methodological advances in saturation, guidelines for its estimation in more complex research designs--such as ethnographic studies--have been lacking. We present an accessible, step-by-step approach to empirical assessment of data saturation, tested on a moderately sized ethnographic study with 109 combined direct observations and…

As in other subfields of psychology, developmental science faces a long-standing problem of limited diversity in research participants. This issue especially raises concerns when researchers make unwarranted broad claims about their results, such as using generic language that implies that a finding is unvarying and applies across participants and…

Marginal maximum likelihood, a common estimation method for item response theory models, is not inherently a Bayesian procedure. However, due to estimation difficulties, Bayesian priors are often applied to the likelihood when estimating 3PL models, especially with small samples. Little focus has been placed on choosing the priors for marginal…

The multistage testing (MST) design has been gaining attention and popularity in educational assessments. For testing programs that have small test-taker samples, it is challenging to calibrate new items to replenish the item pool. In the current research, we used the item pools from an operational MST program to illustrate how research studies…

Recent metascience suggests that motor behavior research may be underpowered, on average. Researchers can perform a priori power analyses to ensure adequately powered studies. However, there are common pitfalls that can result in underestimating the required sample size for a given design and effect size of interest. Critical evaluation of power…

Propensity score matching and weighting methods are often used in observational effectiveness studies to reduce imbalance between treated and untreated groups on a set of potential confounders. However, much of the prior methodological literature on matching and weighting has yet to examine performance for scenarios with a majority of treated…