Prior research in a single clinical training setting has shown Cason and Cason's (1981) simplified model of their performance rating theory can improve rating reliability and validity through statistical control of rater stringency error. Here, the model was applied to clinical performance ratings of 14 cohorts (about 250 students and 200 raters)…

The study of the same people at several occasions is known by various names: panel studies, multi-wave, multi-variable models, cohort analysis, and gain score analysis. The use of econometric concepts, path analysis, and structural equation analysis to study multi-wave, multi-variable models became widespread in methodological practice in the late…

This paper presents a new method for estimating a given latent trait variable by the least-squares approach. The beta weights are obtained recursively with the help of Fourier series and expressed as functions of item parameters of response curves. The values of the latent trait variable estimated by this method and by maximum likelihood method…

Eight different methods are reviewed for determining whether two or more tests are equivalent measures. These methods vary in restrictiveness from the Wilks-Votaw test of compound symmetry (which requires that all means, variances, and covariances are equal), to Joreskog's theory of congeneric tests (which requires only that the tests are measures…

Normative data were obtained on the performance of first-year graduate students on the Aptitude Test and Advanced Tests of the Graduate Record Examinations. The population consisted of students enrolled as full-time graduate students for the first time in the fall of 1964 in a college or university belonging to the Council of Graduate Schools…

A Paradigm for testing concept attainment, comprised of twelve tasks, was formulated. These tasks were hypothesized to form a cumulative hierarchy. Tests were constructed in mathematics and social studies using the paradigm. Data for these tests was analyzed by Kaiser's method for fitting a perfect simplex and Schonemann's method for fitting a…

A summary is provided of the rationale for questioning the applicability of classical reliability measures to criterion referenced tests; an extension of the classical theory of true and error scores to incorporate a theory of dichotomous decisions; a presentation of the mean split-half coefficient of agreement, a single-administration test index…

A cubic spline method for smoothing equipercentile equating relationships under the common item nonequivalent populations design is described. Statistical techniques based on bootstrap estimation are presented for choosing an equating method/degree of smoothing. Smoothing decreases the estimate of random error but results in an increase in…

Argues that rank order correlations may not be ubiquitously suitable for assessing the relations between children's behavior or characteristics at one age or in one situation, and those shown later or in another context. (HOD)

Classifies errors as either systematic or blunder and uncertainties as either systematic or random. Discusses use of error/uncertainty analysis in direct/indirect measurement, describing the process of planning experiments to ensure lowest possible uncertainty. Also considers appropriate level of error analysis for high school physics students'…

The Bookmark Standard Setting Procedure (Lewis, Mitzel, and Green, 1996) is an item-response-theory-based standard setting method that has been widely implemented by state testing programs. The primary purposes of this study were to: (1) estimate standard errors for cutscores that result from Bookmark standard settings under a generalizability…

This study explored possible configurations of the new SAT® critical reading section without analogy items. The item pool contained items from SAT verbal (SAT-V) sections of 14 previously administered SAT tests, calibrated using the three-parameter logistic IRT model. Multiple versions of several prototypes that do not contain analogy items were…

Two sources of inconsistency were separated by reanalyzing data from a major study on short-term consistency. Little evidence was found for generalizability or behavioral predictability. Results supported the assumption that measurement error from short-term fluctuations is not due to systematic individual differences in response consistency.…

Mail questionnaire items that are susceptible to order effects were examined using data from 168 questionnaires in a Florida Cooperative Extension Service evaluation. Order effects were found for multiple-response and attributive questions but not for single-response items. Order also interacted with question complexity, social desirability, and…

Potential methods are explored for removing or otherwise controlling random measurement error, assessment artifacts, irrelevant variation in outcome measures, and confounding sources of covariation in a structural equations model. Using examples with measures of quality of life and functioning, the authors consider these methods for field…