Considers what a responsible test developer would do to gain information to support the consequential basis of validity for a test early in the development. How the consequential basis of validity of the program would be monitored and reported during the life of the program is examined. The validity of the ACT Assessment is considered as if it…

Studied the degree to which equating functions failed to demonstrate population invariance across subpopulations, using two root-mean-square difference measures of the degree to which functions used to link two tests computed on subpopulations differ from the linking function for the whole population. Illustrated the ideas using data from the…

Regular testing can serve as an incentive for students to keep up with their readings, but the time and effort involved in composing and grading frequent tests can serve as an equally strong disincentive to time-strapped instructors. We describe a distorted-item (DI) test that simply requires excerpting sentences or phrases from the assigned…

In optimal assembly of tests from item banks, linear programming (LP) models have proved to be very useful. Assembly by hand has become nearly impossible, but these LP techniques are able to find the best solutions, given the demands and needs of the test to be assembled and the specifics of the item bank from which it is assembled. However,…

Issues pertaining to language and reading while developing and adapting tests are examined. Strengths and limitations associated with the use of readability formulas are discussed. Their use should be confined to paragraphs and longer passages, not items. Readability methods that consider both quantitative and qualitative variables and are…

This article describes the rationale for, and development of, an online instrument that helps identify commonly held science misconceptions. Science Beliefs is a 47-item instrument that targets topics in chemistry, physics, biology, earth science, and astronomy. It utilizes a true or false, along with a written-explanation, format. The true or…

In July 2004 the 36th International Chemistry Olympiad was held in Kiel, Germany. Competition for medals included 236 students from 61 countries, accompanied by about 150 teachers and other mentors. During this Olympiad the students performed qualitative and quantitative analyses of a superconductor, based on lanthanum barium cuprate. In the…

We examined how academic background and course involvement differentially predicted students' performance on lecture- and text-based exam questions (N = 114; 34% men; 76% freshmen). Results showed that academic background and course involvement predicted performance on lecture-based questions and overall exam performance, whereas academic…

This report describes the development, administration, and scoring of open-ended variants of GRE® Subject Test items in biology and psychology. These questions were administered in a Web-based experiment to registered examinees of the respective Subject Tests. The questions required a short answer of 1-3 sentences, and responses were automatically…

This paper provides practical guidance for researchers who are designing studies that randomize groups to measure the impacts of interventions on children. To do so, the paper: (1) provides new empirical information about the values of parameters that influence the precision of impact estimates (intra-class correlations and R-squares); (2)…

Differential item functioning (DIF) analysis can be used to great advantage in second language (L2) assessments. This study examined the differences in performance on L2 test items between groups from different first language backgrounds and suggested ways of improving L2 assessments. The study examined DIF on L2 (Hebrew) test items for two…

The current study compared student performance between paper-and-pencil testing (PPT) and computer-based testing (CBT) on a large-scale statewide end-of-course English examination. Analyses were conducted at both the item and test levels. The overall results suggest that scores obtained from PPT and CBT were comparable. However, at the content…

A thorough understanding of chemical bonding requires familiarity with the particulate nature of matter. In this study, a two-tier multiple-choice diagnostic instrument consisting of ten items (five items involving each of the two concepts) was developed to assess students' understanding of the particulate nature of matter and chemical bonding so…

Posttraumatic stress disorder (PTSD) and major depressive episode (MDE) are frequent and disabling consequences of surviving severe injury. The majority of those who develop these problems are not identified or treated. The aim of this study was to develop and validate a screening instrument that identifies, during hospitalization, adults at high…

Background: Traditional health measurement tools use a pathogenic, or disease origins framework, to assess for the absence of disease or risk factors. Good or positive health, however, is more than the absence of disease and current tools do not reflect this. Purpose: The purpose of this study was to test the psychometric properties of the adult…