Methods of probability modeling to analyze rater agreement are described, emphasizing their basic similarities and viewing them as variants of a common methodology. Statistical techniques for analyzing agreement data are described to address questions such as how many opinions are required to make a medical diagnosis with necessary accuracy. Kappa…

This supplementary report identifies and provides individual descriptions and reviews of 71 retirement forecasting models. Composed of appendices, it is intended as a source of more detailed information than that included in the main volume of the report. Appendix I is an introduction. Appendix II contains individual descriptions of 32 models of…

The aim of this paper is to explore the Rasch model as a criterion of test homogeneity. Two empirical studies are presented to demonstrate this usage. From these studies it is argued that statistical tests of item characteristic curve (ICC) slopes are not sufficient in testing for heterogeneity. Tests of equality of ICC's over groups of subject…

Two of the most popular methods for obtaining equal-interval scales for educational measurement are discussed: Thurstone's method and Item Response Theory (IRT). Between-grade growth on these scales is compared; while unstandardized differences show different trends for the two scales, standardized differences that take standard deviations into…

Parametric measures to estimate J. Cohen's effect size (1966) from a single experiment or for a single study in meta-analysis are investigated. The main objective was to examine the principal statistical properties of this effect size--delta--under variance homogeneity, variance heterogeneity with known variance ratios, and for the Behrens-Fisher…

The estimation of mean and standard errors of the eigenvalues and category quantifications in generalized non-linear canonical correlation analysis (OVERALS) is discussed. Starting points are the delta method equations. The jackknife and bootstrap methods are compared for providing finite difference approximations to the derivatives. Examining the…

It has often been argued that all techniques of standard setting are arbitrary and likely to yield different results for different techniques or persons. This paper deals with a related but hitherto ignored aspect of standard setting, namely, the possibility that Angoff or Nedelsky judges misspecify the probabilities of the borderline student's…

A new test construction method based on integer linear programming is described. This method selects optimal tests in small amounts of computer time. The new method, called the Cluster-Based Method, assumes that the items in the bank have been grouped according to their item information curves so that items within a group, or cluster, are…

Determination of statistical power for analysis of variance procedures requires five elements: (1) significance level; (2) effect size; (3) number of means; (4) error variance; and (5) sample size. Significance levels are traditionally chosen to be 0.5, .01, or .001. Effect size is not discussed in this paper. The number of means is determined by…

The usefulness of a latent trait model designed for use with multidimensional test data was investigated in two stages. The first stage consisted of generating simulation data to fit the multidimensional extension of the two-parameter logistic model, applying the model to the data, and comparing the resulting estimates with the known parameters.…

A review of cross-validation shrinkage formulas is presented which focuses on the theoretical and practical problems in the use of various formulas. Practical guidelines for use of both formulas and empirical cross-validation are provided. A comparison of results using these formulas in a range of situations is then presented. The result of these…

New ability estimators have been proposed by Wainer and Wright (1980) and Mislevy and Bock (1981) that are resistant against guessing and careless behaviors exhibited by some examinees. This paper presents another class of ability estimators that are resistant to departures from the underlying assumptions concerning guessing and carelessness. In…

This study investigated the problem representations formed by college students while solving mathematics problems. Problem representation characteristics indicative of understanding were identified by analyzing audio-tapes and written work of sixteen subjects, ages 16 to 24, who solved mathematics problems using the think-aloud technique. These…

Five methods are given for computing the area of a regular octahedron. It is suggested that students first construct an octahedron as this will aid in space visualization. Six further extensions are left for the reader to try. (LS)