Despite educators' well-founded misgivings about standardized multiple-choice testing, this practice is firmly entrenched in American life. One of educators' major responsibilities is preparing students to make wise choices in school and the "real world." With college and career opportunities hinging on standardized test scores, schools are…

Mail questionnaire items that are susceptible to order effects were examined using data from 168 questionnaires in a Florida Cooperative Extension Service evaluation. Order effects were found for multiple-response and attributive questions but not for single-response items. Order also interacted with question complexity, social desirability, and…

A study of 26 college seniors used introspective and retrospective interviewing to assess cognitive processes used during multiple-choice reading comprehension tests. Categories of processing identified included an overall approach to the test task, reading strategies, and test-taking strategies. (TJH)

Several multiple-choice item formats are examined in the current climate of test reform. The reform movement is discussed as it affects use of the following formats: (1) complex multiple-choice; (2) alternate choice; (3) true-false; (4) multiple true-false; and (5) the context dependent item set. (SLD)

Describes recent efforts of the Center for Research in Evaluation, Standards and Student Testing (CRESST) to evaluate authentic assessment methods, such as portfolios and performance tests. When comparing the merits of authentic versus multiple-choice testing, it is wise to consider validity, reliability, consequences, fairness, generalization,…

Presents a study of the effect of question order on student performance on economics tests. Reports that question order has no statistically significant effect on examination scores, even after including variables that reflect differential human capital characteristics. Concludes that instructors need not worry that some examination versions give…

The Level-11 vocabulary subtest of the Iowa Tests of Basic Skills was analyzed using a two-stage latent trait approach and data set of 2,356 examinees, approximately 11 years of age. It is concluded that the nonparametric approach leads to efficient estimation of the latent trait. (SLD)

The relationship of multiple-choice and free-response items on the College Board's Advanced Placement Computer Science Examination was studied using confirmatory factor analysis. Results with 2 samples of 1,000 high school students suggested that the most parsimonious fit was achieved using a single factor. Implications for construct validity are…

Methods for setting cutoff scores based on criterion performance, normative comparison, and absolute judgment were compared for scores on a multiple-choice psychology examination for 121 undergraduates and 251 undergraduates as a comparison group. All methods fell within the standard error of measurement. Implications of differences for decision…

The consistency of several item bias detection methods was studied across different test administrations of the same items using data from a mathematics test given to approximately 6,600 eighth grade students in all. The Mantel Haenszel and item-response-theory-based sum-of-squares methods were the most consistent. (SLD)

The effect of multiple-choice (MC) or open-ended (OE) response format on diagnostic assessment of algebra test performance was investigated with 231 eighth and ninth graders in Tel Aviv (Israel) using bug or rule space analysis. Both analyses indicated closer similarity between parallel OE subsets than between stem-equivalent OE and MC subsets.…

Effects of schema-based cues (previewing) and structure-based cues (structure cueing) on comprehension of expository prose were studied with 74 fourth graders. Results indicate a significant pattern of higher performance by the previewing group, in comparison with the structure-cued group and a control group. (SLD)

Bias, standard error, and reliability of five ability estimators were evaluated using Monte Carlo estimates of the unknown conditional means and variances of the estimators. Results indicate that estimates based on Bayesian modal, expected a posteriori, and weighted likelihood estimators were reasonably unbiased with relatively small standard…

Investigated the test-taking processes of English-as-a-Foreign-Language (EFL) testees taking a multiple choice listening-comprehension test, evaluating the effectiveness of immediate retrospection for accessing such processes. Linguistic and non-linguistic knowledge was activated in a parallel manner for all subjects with input of various…

Discusses how recent court decisions and the move toward performance assessment may affect the adverse impact, reliability, validity, and pass-fail standards of teacher-certification tests. Recommendations are made for tests that combine multiple-choice items with open-ended tasks. (SLD)