It is demonstrated, theoretically and empirically, that item sets can be selected that meet the unidimensionality assumption of most item response theory models, even though they require more than one ability for a correct response. A method for identifying such item sets for test development purposes is presented. (SLD)

An item response model for multiple-choice items is described and illustrated in item analysis. The model provides parametric and graphical summaries of the performance of each alternative associated with a multiple-choice item. The illustrative application of the model involves a pilot test of mathematics achievement items. (TJH)

Describes a goodness-of-fit statistic that considers the imprecision with which ability is estimated and involves constructing item fit tables based on each examinee's posterior distribution of ability, given the likelihood of the response pattern and an assumed marginal ability distribution. Also describes a Monte Carlo resampling procedure to…

The effect of large, within-examinee item difficulty variability on estimates of the proportion of consistent classification of examinees into mastery categories was studied over 2 test administrations for 100 simulated examinees. The proportion of consistent classifications was adequately estimated using the technique proposed by M. Subkoviak…

A Monte Carlo study, which simulated 10,000 examinees' responses to four tests, investigated the effect of item stratification on parameter estimation in multiple matrix sampling of achievement data. Practical multiple matrix sampling is based on item stratification by item discrimination and a sampling plan with moderate number of subtests. (SLD)

This paper discusses various issues involved in using the Rasch Model with multiple-choice tests and questions the suitability of this model for multiple-choice items. Results of some past studies supporting the model are shown to be irrelevant. The effects of the model's misfit on test equating are demonstrated. (Author JAZ)

Single administration (beta-binomial) estimates for the raw agreement index p and the corrected-for-chance kappa index in mastery testing are compared with those based on two test administrations in terms of estimation bias and sampling variability. Bias is about 2.5 percent for p and 10 percent for kappa. (Author/RL)

Mixed integer linear programing models for customizing two-stage tests are presented. Model constraints are imposed with respect to test composition, administration time, inter-item dependencies, and other practical considerations. The models can be modified for use in the construction of multistage tests. (Author/TJH)

The second response conditional probability model of decision-making strategies used by examinees answering multiple choice test items was revised. Increasing the number of distractors or providing distractors giving examinees (N=106) the option to follow the model improved results and gave a good fit to data for 29 of 30 items. (SLD)

A testlet is an integrated group of test items presented as a unit. The concept of testlet differential item functioning (testlet DIF) is defined, and a statistical method is presented to detect testlet DIF. Data from a testlet-based experimental version of the Scholastic Aptitude Test illustrate the methodology. (SLD)

The phenomenon of simultaneous differential item functioning (DIF) amplification and cancellation and the role of the SIBTEST approach in detecting DIF are investigated with a variety of simulated test data. The effectiveness of SIBTEST is supported, and the implications of DIF amplification and cancellation are discussed. (SLD)

Statistical procedures for measuring heterogeneity of test item distractor distributions, or cultural variation, are presented. These procedures are based on the notion that examinees' responses to the incorrect options of a multiple-choice test provide more information concerning cultural bias than their correct responses. (Author/PN)

This paper develops and illustrates a latent trait approach to constructing an item bank when responses are scored in several ordered categories. This approach is an extension of the methodology developed by Choppin, Wright and Stone, and Wright and Bell for the construction and maintenance of banks of dichotomously scored items. (Author/PN)

An ignored aspect of standard setting, namely the possibility that Angoff or Nedelsky judges specify inconsistent probabilities (e.g., low probabilities for easy items but large probabilities for hard items) is explored. A latent trait method is proposed to estimate such misspecifications, and an index of consistency is defined. (Author/PN)

Presents logistic discriminant analysis as a means of detecting differential item functioning (DIF) in items that are polytomously scored. Provides examples of DIF detection using a 27-item mathematics test with 1,977 examinees. The proposed method is simpler and more practical than polytomous extensions of the logistic regression DIF procedure.…