Men tend to achieve higher response accuracy than women on surveys of political knowledge. We investigated the possibility that this performance gap is moderated by factors that render the communicative context of a survey intellectually threatening to women and thereby induce stereotype threat. In a telephone survey of college students' political…

Test-item writing efforts typically results in item pools with an undesirable correlational structure between the content attributes of the items and their statistical information. If such pools are used in computerized adaptive testing (CAT), the algorithm may be forced to select items with less than optimal information, that violate the content…

The purpose of this study was to examine the relations of various cognitive abilities and aspects of math performance with computational estimation skill among third graders. Students (n=315) were assessed on language, nonverbal reasoning, concept formation, processing speed, long-term memory, working memory, inattentive behavior, basic reading…

This study investigated factors related to score differences on computerized and paper-and-pencil versions of a series of primary K-3 reading tests. Factors studied included item and student characteristics. The results suggest that the score differences were more related to student than item characteristics. These student characteristics include…

This article considers and illustrates a strategy to study effects of school context on differential item functioning (DIF) in large-scale assessment. The approach employs a hierarchical generalized linear modeling framework to (a) detect DIF, and (b) identify school-level correlates of the between-group differences in item performance. To…

Data were extracted from the Early Childhood Longitudinal Study (ECLS), a national sample of more than 20,000 kindergartners and first-graders. Fifty-one social studies and science test items were combined into a General Knowledge Test. This test was individually administered to each child with no reading required. General Knowledge Test scores…

A cognitive lab technique (n=156) was used to investigate interactions between individual factors and item factors presumed to affect assessment validity for diverse students, including English language learners. Findings support the concept of "access"--an interaction between specific construct-irrelevant item features and individual…

This study, a partial replication of Bruhn de Garavito (1999a; 1999b), investigates the second language (L2) acquisition of Spanish reflexive passives and reflexive impersonals by French- and English-speaking adults at an advanced level of proficiency. The L2 acquisition of Spanish reflexive passives and reflexive impersonals by native French and…

Whether the presence of bidimensionality has any effect on the adaptive recalibration of test items was studied through live-data simulation of computer adaptive testing (CAT) forms. The source data were examinee responses to the 298 scored multiple choice items of a licensure examination in a health care profession. Three 75-item part-forms,…

The primary goal of this project was to examine the predictability of Scholastic Aptitude Test (SAT) reading item difficulty (equated delta) for main idea items, and the predictability of main idea, inference, and explicit statement item types. A secondary purpose was to contrast the responses of high verbal and low verbal ability examinees.…

Contrasts between constructed response items and stem-equivalent multiple-choice counterparts typically have involved averaging item characteristics, and this aggregation has masked differences in statistical properties at the item level. Moreover, even aggregated format differences have not been explained in terms of differential cognitive…

In order to evaluate the Test of English as a Foreign Language (TOEFL) vocabulary item format and to determine the effectiveness of alternative vocabulary test items, this study investigated the functioning of eight different multiple-choice formats that differed with regard to: (1) length and inference-generating quality of the stem; (2) the…

Items with the highest discrimination parameter values in a logistic item response theory (IRT) model do not necessarily give maximum information. This paper shows which discrimination parameter values (as a function of the guessing parameter and the distance between person ability and item difficulty) give maximum information for the…

The relationship between test item characteristics and testing time was studied for a computer-administered licensing examination. One objective of the study was to develop a model to predict testing time on the basis of known item characteristics. Response latencies (i.e., the amount of time taken by examinees to read, review, and answer items)…

This study examined the relationship of a machine-scorable, constrained free-response computer science item that required the student to debug a faulty program to two other types of items: multiple-choice and free-response requiring production of a computer program. The free-response items were from the College Board's Advanced Placement Computer…