Given the potential value of reliability generalization (RG) studies in the development of cumulative psychometric knowledge, the purpose of this paper is to provide a tutorial on how to conduct such studies and to serve as a guide for researchers wishing to use this methodology. After some brief comments on classical test theory, the paper…

Test validity refers to the degree to which the inferences based on test scores are meaningful, useful, and appropriate. Thus, test validity is a characteristic of a test when it is administered to a particular population. This article introduces the modern concepts of validity advanced by S. Messick (1989, 1996, 1996). Traditionally, the means of…

Replies to Pauk's investigation, raising some questions about it and defending his own readability measure. (MD)

The gross-motor activity of 27 three- and four- year-olds was assessed through teacher ratings, parent responses to the activity scale of the Colorado Childhood Temperament Inventory, and data from uncalibrated actometers worn by children during free play. Activity scores composited across multiple actometers had high reliability and correlated…

Reveals that survey respondents who agree to complete time-use diaries differ significantly in several characteristics from those respondents who are only willing to complete a questionnaire. (FL)

The Mean Cost Rating, P(A) from Signal Detection Theory, Kendall's rank correlation coefficient tau, and Goodman and Kruskal's gamma measures of predictive power are compared and shown to be different transformations of the statistic S. Gamma is generally preferred for hypothesis testing. Measures of association for ordered contingency tables are…

The Megargee and Dorhout classificatory rules were applied to 181 MMPI profiles from inmates at a Level 4 Federal Correctional Institution. Fifty of these inmates were retested and reclassified. Only 14 of these 50 inmates retained their original type designation upon retesting. (Author)

In a classroom demonstration of reliability concepts, 65 introductory psychology students each measured 50 lines of 25 different lengths. Student measurement were compared to actual lengths and then tabulated for mean, median, mode, and standard deviation. Even with a small sample, findings supported classical reliability theory. (AM)

Competence is seen as a cause of effective performance, not a synonym for it. Teaching and assessing competence may be different than teaching and assessing academic skills. Competence can be measured, but its measurement depends first on its definition. (Author/MLW)

The effectiveness of using video replays to judge gymnastics vaulting was examined by determining reliability of judges' scores and comparing mean scores for original (live) vaults and replays. Statistical analyses suggest that variability of scores was greater after the live vault, and that judges' reliability may be enhanced by replays.…

Studies in applied behavior analysis have used two expressions of reliability for human observations: percentage agreement and correlational techniques (including the phi coefficient). Formulas for converting percentage agreement scores to phi coefficients and vice versa are presented. (Author/RL)