Cohen's kappa coefficient is commonly used for assessing agreement between classifications of two raters on a nominal scale. Three variants of Cohen's kappa that can handle missing data are presented. Data are considered missing if one or both ratings of a unit are missing. We study how well the variants estimate the kappa value for complete data…

In this study, we examined the results and interpretations produced from two different IRTree models--one using paths consisting of only dichotomous decisions, and one using paths consisting of both dichotomous and polytomous decisions. We used data from two versions of an impulsivity measure. In the first version, all the response options had…

In 1881, Donald MacAlister posed a problem in the "Educational Times" that remains relevant today. The problem centers on the statistical evidence for the effectiveness of a treatment based on a comparison between two proportions. A brief historical sketch is followed by a discussion of two default Bayesian solutions, one based on a…

An alternative to null hypothesis significance testing is presented and discussed. This approach, referred to as observation-oriented modeling, is centered on model building in an effort to explicate the structures and processes believed to generate a set of observations. In terms of analysis, this novel approach complements traditional methods…

Cluster randomized trials involving participants nested within intact treatment and control groups are commonly performed in various educational, psychological, and biomedical studies. However, recruiting and retaining intact groups present various practical, financial, and logistical challenges to evaluators and often, cluster randomized trials…

To further understand the properties of data-generation algorithms for multivariate, nonnormal data, two Monte Carlo simulation studies comparing the Vale and Maurelli method and the Headrick fifth-order polynomial method were implemented. Combinations of skewness and kurtosis found in four published articles were run and attention was…

In this study, smoothing and scaling approaches are compared for estimating subscore-to-composite scaling results involving composites computed as rounded and weighted combinations of subscores. The considered smoothing and scaling approaches included those based on raw data, on smoothing the bivariate distribution of the subscores, on smoothing…

A number of psychometricians have argued for the use of parallel analysis to determine the number of factors. However, parallel analysis must be viewed at best as a heuristic approach rather than a mathematically rigorous one. The authors suggest a revision to parallel analysis that could improve its accuracy. A Monte Carlo study is conducted to…

For the past 25 years, methodological advances have been made in missing data treatment. Most published work has focused on missing data in dependent variables under various conditions. The present study seeks to fill the void by comparing two approaches for handling missing data in categorical covariates in logistic regression: the…

Quick power estimates for detecting a difference in two population proportions are expedient during early stages of research planning. Such estimates are tabled and graphed in this article. They are shown to be conservative but accurate for most research situations when proportions fall in the range of 0.25 to 0.75. (SLD)

Coombs' formulation of Data Quadrants, and Runkle and McGrath's adaptation, were found to be incomplete. The third dichotomy was nonfunctional in half the Quadrants, resulting in inconsistent interpretation of the proximity element. The category "comparison between points and dyads" should be added to complete the logical classification…

The procedures required to obtain the approximate posterior standard deviations of the parameters in the three commonly used item response models for dichotomous items are described and used to generate values for some common situations. The results were compared with those obtained from maximum likelihood estimation. It is shown that the use of…

Sixteen measures of reliability for two-category nominal scales are compared. Upon correcting for chance agreement, there are only five distinct indices: Fleiss's modification of A-sub-1, the phi coefficient, Cohen's kappa, and two intraclass coefficients. Recommendations for choosing an agreement index are made based on definitions, magnitude,…

The superior program proved to be suprogram PROB published by the National Bureau of Standards. (Author)

Complete paired comparisons data was obtained by use of multiple rank order (MRO) in the context of gathering rank order preferences of grade six students, their parents, and teachers for instructional objectives. Partially balanced incomplete block designs with two associate classes were used in the MRO instruments instead of the usual balanced…