This study is an attempt at a cohesive characterization of the concept of essay reliability. As such, it takes as a basic premise that previous and current practices in reporting reliability estimates for essay tests have certain shortcomings. The study provides an analysis of these shortcomings--partly to encourage a fuller understanding of the…

Scores from the Stanford Achievement Tests administered to 50,000 students in Palm Beach County, Florida, were studied in order to determine whether the speeded nature of the reading comprehension subtest was related to inconsistencies in the score profiles. Specifically, the probable effect of random guessing was examined. Reading scores were…

This paper describexs the difficulties encountered in developing a teacher merit pay system based on pretest-posttest class gain scores. The different stages of development are scrutinized, from the initial stage of achievement test selection through the specification of a criterion score for superior teacher productivity. The pretest-posttest…

A study assessed the relative effectiveness of four types of follow-up techniques in increasing responses to mailed questionnaires requiring responses from multiple individuals. The respondents were college administrators. Copies of questionnaires designed to measure professional development were mailed to a random sample of 693 chief student…

A scoring rubric transition study is described. It was designed to evaluate possible drift in scoring the Maryland Writing Test from year to year (when using a modified holistic scoring method), to evaluate strategies for revising swing rubrics from narrative and explanatory writing while maintaining original scoring standards, and to establish…

The document presents findings from a comprehensive review of the literature on the topic of nonbiased assessment. An introductory chapter describes the review's conceptual framework. Chapters 2 through 9 present analyses of the following major aspects of the topic (sample subtopics in parentheses): historical perspectives (ancient influences,…

Sixty-three students in three undergraduate college classes were divided into three groups to investigate the effect of two delayed self-reporting techniques on attitudinal data toward a course and instructor. Nonparametric tests and intercorrelations of item treatment means, together with two-way analyses of variance on item clusters, indicated…

If a loss function is available specifying the social cost of an error of measurement in the score on a unidimensional test, an asymptotic method, based on item response theory, is developed for optimal test design for a specified target population of examinees. Since in the real world such loss functions are not available, it is more useful to…

The validity of combining a correction for guessing with the Nedelsky-based cutscore was investigated. A five option multiple choice Mathematics Achievement Test was used in the study. Items were selected to meet several criteria. These included: the capability of measuring mathematics concepts related to performance in introductory statistics;…

To show the difficulty of eliminating test bias and to develop a methodology for distinguishing between the effects of prior knowledge and of skill development on reading comprehension, 207 eighth grade students from rural and urban areas were administered an 18-question reading comprehension test. Quantitative and qualitative effects of prior…

Efforts to study the fidelity of translation of attitudinal scales into foreign languages have faltered due to the lack of powerful statistical tests to assess such transformations. This study uses a maximum likelihood factor analysis procedure to compare multivariate factor structures across subpopulations. The results showed that inconsistent…

Differences between four test-independent components of testwiseness and their relative importance were studied in a quasi-experimental investigation of their effects. The components were: (1) time-using training; (2) error-avoidance training; (3) guessing training; and (4) deductive-reasoning training. Three parallel forms of a 30-item test were…

A method is proposed to equate different sets of items administered to different groups of individuals using the Rasch model. A Rasch equating model was formulated to describe one common Rasch scale in different groups with different but overlapping sets of items. The item parameters can then be estimated simultaneously, avoiding different…

A computerized flexilevel test was implemented and its ability estimates were compared with those of a Bayesian estimation based computerized adaptive test (CAT) as well as with known true ability estimates. Results showed that when the flexilevel test was terminated according to Lord's criterion, its ability estimates were highly and…

The study reported in this paper examined whether a disproportionate number of boys and girls in a given grade level at various schools might have an inequitable effect on the mean achievement test scores of those schools. The student population of the Palm Beach County Schools, Florida, reflecting a cross-section of socio-economic backgrounds,…