When low-stakes assessments are administered, the degree to which examinees give their best effort is often unclear, complicating the validity and interpretation of the resulting test scores. This study introduces a new method, based on item response time, for measuring examinee test-taking effort on computer-based test items. This measure, termed…

Study 1 assessed students' use and perceptions of acronyms at 3 different exam times in 2 sections of Introduction to Psychology. Acronym use consistently predicted higher performance on acronym related exam items, and I partially discounted 2 possible confounds. Students rated acronyms as helpful in multiple ways, including increasing motivation…

In the Spring 2005 issue, Wang, Schnipke, and Witt provided an informative description of the task inventory approach that centered on four functions of job analysis. The discussion included persuasive arguments for making systematic connections between tasks and KSAs. But several other facets of the discussion were much less persuasive. This…

We examined the validity of the comprehension component of the Gray Oral Reading Test (GORT; Wiederholt & Bryant, 1992, 2001) by assessing whether reading really is required to answer its questions. The extent to which GORT questions are passage independent was assessed by having participants answer them without reading the passages. Most…

To meet the requirements of the No Child Left Behind Act, school districts and states must compile summary reports of the levels of student achievement in reading and mathematics. The levels are to be described in broad categories: "basic and below," "proficient," or "advanced." Educational units are given considerable latitude in defining the…

The person-response function (PRF) relates the probability of an individual's correct answer to the difficulty of items measuring the same latent trait. Local deviations of the observed PRF from the expected PRF indicate person misfit. We discuss two new approaches to investigate person fit. The first approach uses kernel smoothing to estimate…

This study describes the relation between personality items' validities, defined as the items' correlations with acquaintance ratings on the Big 5 personality factors, and other itemmetric properties including ambiguity, syntactic complexity, social desirability, content, and trait indicativity. Five external validity coefficients for each item on…

This study addressed what standard-setting panelists think about when they make item performance estimates for a barely proficient student. This study extended previous studies by considering the factors that influenced panelists' decisions in an Angoff (1971)-based standard-setting study as a function of their item performance estimates.…

To test preschoolers' ability to flexibly switch between abstract rules differing in difficulty, ninety-three 3-, 4-, and 5-year-olds were instructed to switch from an (easier) shape-sorting to a (harder) function-sorting rule, or vice versa. Children learned one rule, sorted four test sets, then learned the other rule, and sorted four more sets.…

This research explored the absolute accuracy of judgments of learning (JOLs), wherein absolute accuracy pertains to how well the magnitude of the participant's predictions of recall correspond to his or her subsequent recall. The Anchoring Hypothesis proposes that the magnitude of JOLs does not change systematically with item difficulty; analogous…

Mathematical literacy is an important skill that is gaining the attention of mathematics educators. Students are increasingly challenged on standardized assessments to read, create, use, and comprehend numerous mathematical representations as a way of demonstrating mathematical literacy. Test items assessing algebra concepts from the Texas…

The use of "neutral or unsure" on an instrument designed to measure identity development in college students was explored. The nominal response model from item response theory was used to evaluate whether "neutral or unsure" was used more frequently by those at low or middle levels of development; the results depended on the subscale and sometimes…

The purpose of this article is to present a working definition of the term "culture," as well as to describe and demonstrate a comprehensive framework for evaluating hypotheses about cultural bias in educational testing. The framework is demonstrated using 5th-grade reading and language arts data from the Terra Nova test (CTB/McGraw-Hill, 1999).…

Recent studies have shown that restricting review and answer change opportunities on computerized adaptive tests (CATs) to items within successive blocks reduces time spent in review, satisfies most examinees' desires for review, and controls against distortion in proficiency estimates resulting from intentional incorrect answering of items prior…

In today's higher education, high quality assessments play an important role. Little is known, however, about the degree to which assessments are correctly aimed at the students' levels of competence in relation to the defined learning goals. This article reviews previous research into teachers' and students' perceptions of item difficulty. It…