There is a need for effect sizes that are readily interpretable by a broad audience. One index that might fill this need is [pi], which represents the proportion of scores in one group that exceed the mean of another group. The robustness of estimates of [pi] to violations of normality had not been explored. Using simulated data, three estimates…

BASIE is a framework for interpreting impact estimates from evaluations. It is an alternative to null hypothesis significance testing. This guide walks researchers through the key steps of applying BASIE, including selecting prior evidence, reporting impact estimates, interpreting impact estimates, and conducting sensitivity analyses. The guide…

One important issue in Bayesian estimation is the determination of an effective informative prior. In hierarchical Bayes models, the uncertainty of hyperparameters in a prior can be further modeled via their own priors, namely, hyper priors. This study introduces a framework to construct hyper priors for both the mean and the variance…

The use of p-values in combining results of studies often involves studies that are potentially aberrant. This paper proposes a combined test that permits trimming some of the extreme p-values. The trimmed statistic is based on an inverse cumulative normal transformation of the ordered p-values. (SLD)

This study compared the model of J. E. Grizzle, C. F. Starmer, and G. G. Koch (GSK, 1969) and log-linear model-based approaches for testing hypotheses in r x c contingency tables. Tables were simulated under various conditions of table, sample, row-effect size, and column-effect size. Test statistics for column (main) and interaction effects were…

This paper is concerned with the construction of effect size standard errors in situations where the effect sizes are independent but the data have likely been sampled from non-normal distributions, and possibly for different studies, from different families of non-normal distributions. Asymptotic distribution-free estimators are provided for two…

This paper discusses distribution theory and power computations for four common "tests of combined significance." These tests are calculated using one-sided sample probabilities or p values from independent studies (or hypothesis tests), and provide an overall significance level for the series of results. Noncentral asymptotic sampling…