This study examined the SAT® II: Writing Subject Test as a predictor of writing performance in college English courses. Writing performance was based on eight writing samples submitted as part of regular course work by students in eight colleges. The samples consisted of drafts and final papers submitted in response to four take-home writing…

Over 2,000 writing samples were collected from four undergraduate institutions and compared, where possible, with scores on a multiple-choice test. High correlations between ratings of the writing samples and multiple-choice test scores were obtained. Samples contributed substantially to the prediction of both college grades and writing…

Presents the results of an investigation to determine if the position of a distractor in a multiple choice question influences the degree of difficulty of an item. The data support the hypothesis that the placement of the distractor immediately before the key alters the difficulty of the item significantly. (Authors/SA)

The California Commission on Teacher Credentialing's new program standards call for improved assessments of administrative performance. A wide range of traditional and nontraditional assessment approaches are being developed, including interviews, multiple-choice examinations, written or constructed responses, performance objectives and…

Despite educators' well-founded misgivings about standardized multiple-choice testing, this practice is firmly entrenched in American life. One of educators' major responsibilities is preparing students to make wise choices in school and the "real world." With college and career opportunities hinging on standardized test scores, schools are…

Mail questionnaire items that are susceptible to order effects were examined using data from 168 questionnaires in a Florida Cooperative Extension Service evaluation. Order effects were found for multiple-response and attributive questions but not for single-response items. Order also interacted with question complexity, social desirability, and…

A study of 26 college seniors used introspective and retrospective interviewing to assess cognitive processes used during multiple-choice reading comprehension tests. Categories of processing identified included an overall approach to the test task, reading strategies, and test-taking strategies. (TJH)

Several multiple-choice item formats are examined in the current climate of test reform. The reform movement is discussed as it affects use of the following formats: (1) complex multiple-choice; (2) alternate choice; (3) true-false; (4) multiple true-false; and (5) the context dependent item set. (SLD)

Describes recent efforts of the Center for Research in Evaluation, Standards and Student Testing (CRESST) to evaluate authentic assessment methods, such as portfolios and performance tests. When comparing the merits of authentic versus multiple-choice testing, it is wise to consider validity, reliability, consequences, fairness, generalization,…

Presents a study of the effect of question order on student performance on economics tests. Reports that question order has no statistically significant effect on examination scores, even after including variables that reflect differential human capital characteristics. Concludes that instructors need not worry that some examination versions give…

The Level-11 vocabulary subtest of the Iowa Tests of Basic Skills was analyzed using a two-stage latent trait approach and data set of 2,356 examinees, approximately 11 years of age. It is concluded that the nonparametric approach leads to efficient estimation of the latent trait. (SLD)

The relationship of multiple-choice and free-response items on the College Board's Advanced Placement Computer Science Examination was studied using confirmatory factor analysis. Results with 2 samples of 1,000 high school students suggested that the most parsimonious fit was achieved using a single factor. Implications for construct validity are…

Methods for setting cutoff scores based on criterion performance, normative comparison, and absolute judgment were compared for scores on a multiple-choice psychology examination for 121 undergraduates and 251 undergraduates as a comparison group. All methods fell within the standard error of measurement. Implications of differences for decision…

The consistency of several item bias detection methods was studied across different test administrations of the same items using data from a mathematics test given to approximately 6,600 eighth grade students in all. The Mantel Haenszel and item-response-theory-based sum-of-squares methods were the most consistent. (SLD)

The effect of multiple-choice (MC) or open-ended (OE) response format on diagnostic assessment of algebra test performance was investigated with 231 eighth and ninth graders in Tel Aviv (Israel) using bug or rule space analysis. Both analyses indicated closer similarity between parallel OE subsets than between stem-equivalent OE and MC subsets.…