This article considers the design process for new formative history assessments. Over the course of 3 years, my colleagues from the Stanford History Education Group and I designed, piloted, and revised dozens of "History Assessments of Thinking" (HATs). As we created HATs, we sought to gather information about their cognitive validity,…

Linguistic complexity of test items is one test format element that has been studied in the context of struggling readers and their participation in paper-and-pencil tests. The present article presents findings from an exploratory study on the potential relationship between linguistic complexity and test performance for deaf readers. A total of 64…

There is much debate on the merits and pitfalls of standardized tests for college admission, with questions regarding the format (multiple-choice versus constructed response), cognitive complexity, and content of these assessments (achievement versus aptitude) at the forefront of the discussion. This study addressed these questions by…

Studied content domain, format, and level of inquiry as factors contributing to the large variation in student performance across open-ended measures. Results for more than 1,200 eighth graders do not support the hypothesis that tasks similar in content, format, and level of inquiry would correlate higher with each other than with measures…

Gender differences on the National Education Longitudinal Study of 1988 science tests were explored through statistical analyses and interviews with 25 high school students. Results show the importance of studying the validity of the outcome measure and suggest that conclusions about group differences and correlates of achievement depend on the…

A study was conducted to investigate: (1) the relationships between the results from various forms of assessment and the patterns of correlation across content areas; (2) how cognitive components correlate with the test results from different classroom assessments; and (3) how content areas affected the relationships. Data were collected from a…

This study investigated three factors that may contribute to the large variation in student performance across open-ended measures. These factors are content domain, format (whether the task required only pencil and paper or involved a hands-on manipulation of equipment), and level of inquiry (whether the task guided the student toward the…

Data from the College Board Validity Study Service show that the average multiple correlation of the Scholastic Aptitude Test (SAT) with college grades peaked in 1974 and then tended to decline. Data from other sources also estimate a small average decline from 1974 to 1985. This study documented changes in the SAT and related these changes to…

The idea that validity should be considered a property of inferences, rather than of assessments, has developed slowly over the past century. In early writings about the validity of educational assessments, validity was defined as a property of an assessment. The most common definition was that an assessment was valid to the extent that it…

This study investigated the comparability of scores for paper and computer versions of a writing test administered to eighth grade students. Two essay prompts were given on paper to a nationally representative sample as part of the 2002 main NAEP writing assessment. The same two essay prompts were subsequently administered on computer to a second…