Methods for evaluating the consistency of responses to test items were compared. When a researcher is unwilling to make the assumptions of classical test theory, has only a small number of items, or is in a tailored testing context, Cliff's dominance indices may be useful. (Author/CTM)

Some practical guidelines for developing multiple choice tests are offered. Included are three steps: (1) test design; (2) proper construction of test items; and (3) item analysis and evaluation. (JMF)

The National Staff Development Council (NSDC), a private, nonprofit association, has outlined high standards for educator professional learning. One demonstration of NSDC's commitment to the goal of ensuring all schools support and use high standards for professional learning is the organization's investment in developing an instrument to assess…

A modification of Hoyt's analysis of variance model for test analysis was proposed by Lu. A difficulty that may be encountered in using Lu's modification is examined, and a solution is proposed. (JKS)

There is variability in the estimation of an equating transformation because common-item parameters are obtained from responses of samples of examinees. The most commonly used standard error of equating quantifies this source of sampling error, which decreases as the sample size of examinees used to derive the transformation increases. In a…

An estimate and upper-bound estimate for the reliability of a test composed of binary items is derived from the multidimensional latent trait theory of R. D. Bock and M. Aitken (1981). The practical uses of such estimates are discussed. (SLD)

Prior computer expertise represents one of the most important predictors of performance when interacting with ICT (Information and Communication Technologies) and acquiring computer skills. Due to demographic changes, the older adult will become increasingly important as a potential user. However, there is a lack of instruments for the assessment…

In this paper, formulas for computing the weights that maximize the reliability of a test with multiple parts are derived using a congeneric model. A direct derivation for the three-part test and case and a two-step derivation for the n-part case are presented, and results for these two approaches are shown to be consistent for the three-part…

Contrary to common belief, reliability estimates of number-right multiple-choice tests are not inflated by speededness. Because examinees guess on questions when they run out of time, the responses to these questions show less consistency with the responses of other questions, and the reliability of the test will be decreased. The surprising…

A major deficiency in classical test theory is the reliance on Pearson product-moment (PPM) correlation concepts in the definition of reliability. PPM measures are totally insensitive to first moment differences in tests which leads to the dubious assumption of essential tan-equivalence. Robinson proposed a measure of agreement that is sensitive…

Hoyt's analysis of variance procedure for estimating reliability assumes that the residual mean square estimates error variability. If, however, an individual's true score varies across items, it is argued that residual mean square estimates two components--error and interaction--and hence Winer's modification of Hoyt's formula, understimates the…

Option weighting is an alternative to increasing test length as a means of improving the reliability of a test. The effects on test reliability of option weighting procedures were compared in two empirical studies using four independent sets of items. Biserial weights were found to be superior. (Author/CTM)

A paradoxical phenomenon of decreases in reliability as the number of elements averaged over increases is shown to be possible in multifacet reliability procedures (intraclass correlations or generalizability coefficients). Conditions governing this phenomenon are presented along with implications and cautions. (Author)

Most psychological tests and inventories, particularly in personality and attitude measurement, contain common items. The present report specifically considered common-item contributions to test internal consistency reliability using extant data from the California Psychological Inventory. No negative contribution of item overlap was found.…

Summarizes the benefits of computerized assessment and provides a review of some practical issues concerning measurement, item and examinee characteristics, hardware, and software. Adequate measures of reliability and validity have been established for many computer-based tests, and the benefits of computer testing have been realized in applied…