Effects of test length, sample size, and assumed ability distribution were investigated in a multiple replication Monte Carlo study under the 1-parameter (1P) and 2-parameter (2P) logistic graded model with five score levels. Accuracy and variability of item parameter and ability estimates were examined. Monte Carlo methods were used to evaluate…

The purpose of this study is to help define the precise nature and limits of the tolerable range in which a researcher may be relatively confident about the statistical validity of his or her research findings, focusing specifically on the statistical validity of results when violating the assumptions associated with the one-way, fixed-effects…

Rasch between and total weighted and unweighted fit statistics were compared using varying test lengths and sample sizes. Two test lengths (20 and 50 items) and three sample sizes (150, 500, and 1,000 were crossed. Each of the six combinations were replicated 100 times. In addition, power comparisons were made. Results indicated that there were no…

This study of the validity of the Graduate Record Examinations (GRE) General Test used data from predictive validity studies that were conducted by the GRE Validity Study Service (VSS) in 79 graduate departments. The performance criterion was first-year grades in graduate school. Observed validities were computed, and for each graduate department…

Monte Carlo methods were employed to assess the relative power of the paired samples t test and Wilcoxon's signed-ranks test under ten population shapes. Results of the study indicated that: (1) each of the two statistics was more powerful than the other in given situations; (2) the power advantages of the t test under normal theory were small;…

The use of three remedies for errors in the measurement of ability that arise from differences in rater stringency is discussed. Models contrasted are: (1) Conventional; (2) Handicap; and (3) deterministic Rater Response Theory (RRT). General model requirements, power, bias of measures, computing cost, and complexity are contrasted. Contrasts are…

A total of 2,000 essays written by 1,000 students was submitted to generalizability analyses for domain-referenced tests. Each student had written one essay on each of two prompts representing two models of discourse. Each essay was read by six readers and judged on a scale of from 1 to 4. No reader read essays from both prompts. Reader agreement…

Five comparisons were made relative to the quality of estimates of ability parameters and item calibrations obtained from the one-parameter and three-parameter logistic models. The results indicate: (1) The three-parameter model fit the test data better in all cases than did the one-parameter model. For simulation data sets, multi-factor data were…

Multiple matrix sampling is a sampling design in which both test items and examinees are randomly sampled from their respective populations. This study was designed to develop and assess a method for computing an estimate of a correlation coefficient when a multiple matrix sampling design is used. The examinee populations included 212 third-grade…

In most reliability studies, the precision of a reliability estimate varies inversely with the number of examinees (sample size). Thus, to achieve a given level of accuracy, some minimum sample size is required. An approximation for this minimum size may be made if some reasonable assumptions regarding the mean and standard deviation of the test…

The power of the classical Linear Discriminant Function (LDF) is compared, using Monte Carlo techniques with five other procedures for classifying observations from certain non-normal distributions. The alternative procedures considered are the Quadratic Discriminant Function, a Nearest Neighbor Procedure with Probability Blocks, and three density…

Responds to commentaries by Bereiter and Gersten concerning findings of the High/Scope Educational Research Foundation's 15-year Preschool Curriculum Comparison Study. Addresses issues of design of study, experimenter bias, sample size, self-report validity, interpretation of findings, and related topics. (NH)

The influences of class size and the number of students completing surveys on faculty and course ratings were studied. For classes of 30 or more, a 50 percent response rate gives an acceptable indication of rating, while for a class of less than 30, about 80 percent return is needed. (Author/SW)

The comparison of item response theory models for the Test of English as a Foreign Language (TOEFL) was extended to an equating context as simulation trials were used to "equate the test to itself." Equating sample data were generated from administration of identical item sets. Equatings that used procedures based on each model (simple…

The development of the new fifth edition of the Sixteen Personality Factor Questionnaire (16PF) is described. The factor structure of the new 16PF was explored with 4 samples ranging from 646 to 3,498 subjects. Results support the validity of the 16PF factor structure and its continuity with earlier versions. (SLD)