Computer simulation indicated that reliability increased with the degree of similarity between underlying and observed distributions when the observed categorical distribution was deliberately constructed to match the shape of the underlying distribution of the trait being measured. Reliability also increased with correlation among variables and…

Causes of local item dependence in a large-scale performance assessment were studied using data from the Maryland School Performance Assessment Program. Contextual characteristics (content and response requirements) were identified to differentiate locally independent and dependent item clusters. Hypothesized explanations are offered for high…

Examined the effects of test length and sample size on the alternate forms reliability and equating of simulated mathematics tests composed of constructed response items scaled using the two-parameter partial credit model. Results suggest that, to obtain acceptable reliabilities and accurate equated scores, tests should have at least 8 6-point…

Over 5,000 students participated in a study of the dimensionality and stability of the item parameter estimates of a mathematics performance assessment developed for the Quantitative Understanding: Amplifying Student Achievement and Reasoning (QUASAR) Project. Results demonstrate the test's dimensionality and illustrate ways to examine use of the…

The contribution of item response theory to the validity of interpretations of achievement test results is reviewed in the context of four applications. The applications include construction of scales for achievement tests, test construction, development of customized tests, and investigation of the influence of instruction on achievement tests.…

Three procedures for the detection of differential item functioning under item response theory were compared. Data for 2 forms of a mathematics test taken by 1,490 college students were analyzed through F. M. Lord's chi-square, N. S. Raju's area measures, and the likelihood ratio test. (SLD)

Describes examinee-centered method for setting multiple cutscores on a test involving both objective and extended-response items. Judges evaluate a representative sample of examinee performance using a rating scale that is defined in terms of performance standards, and these ratings are linked to examinee's test scores to generate a functional…

Effects of altering test difficulty on examinee ability measures and test length in a computer adaptive test were studied for 225 medical technology students in 3 test difficulty conditions. Results suggest that, with an item pool of sufficient depth and breadth, acceptable targeting to test difficulty is possible. (SLD)

A Many-Faceted Rasch Model (FACETS) for measurement of writing ability is described, and its use in solving measurement problems in large-scale assessment is illustrated with a random sample of 1,000 students from Georgia's Eighth Grade Writing Test. It is a promising approach to assessment through written compositions. (SLD)