Between administrations of a test, 36 nursing students were given a practice test and answer key that provided feedback; 30 were not. Those who performed poorly on the test were more overconfident about answers to hard questions. This judgment bias can be reduced by providing feedback about their performance and confidence. (Contains 54…

Reports on the potential problems in scoring responses to sequencing tests, the development of a computer program to overcome these difficulties, and an exploration of the value of scoring procedures. (Author/VWL)

Discusses an approach to analyzing performance assessments that identifies potential reasons for misfitting items and uses this information to improve on items and rubrics for these assessments. Illustrates the approach through a 53-item mathematics performance assessment completed by approximately 500 middle school students. (SLD)

Compares Angoff and Nedelsky methods for setting passing scores on tests. Using one of the methods, 84 college students were taught to estimate their probable scores on a vocabulary test. Estimates were compared to their later actual scores. The Nedelsky method was considerably less accurate under certain conditions, and both methods…

Proposes an item-selection algorithm for neutralizing the differential effects of time limits on computerized adaptive test scores. Uses a statistical model for distributions of examinees' response times on items in a bank that is updated each time an item is administered. Demonstrates the method using an item bank from the Armed Services…

Investigated three models that permit restricted examinee control over revising previous answers in the context of adaptive testing, using simulation. Two models permitting item revisions worked well in preserving test fairness and accuracy, and one model may preserve some cognitive processing styles developed by examinees for a linear testing…

Examined whether patterns of gender differential item functioning (DIF) in parcels of items are influenced by changes in item position. Findings for more than 2,000 college freshmen taking a test of mathematics suggest that the amounts of gender DIF and DIF present in item parcels tend not to be influenced by changes in item position. (SLD)

A well-constructed test sheet not only helps the instructor evaluate the learning status of the students, but also facilitates the diagnosis of the problems embedded in the students' learning process. This paper addresses the problem of selecting proper test items to compose a test sheet that conforms to such assessment requirements as average…

When the item response theory (IRT) model uses the marginal maximum likelihood estimation, person parameters are usually treated as random parameters following a certain distribution as a prior distribution to estimate the structural parameters in the model. For example, both PARSCALE (Muraki & Bock, 1999) and BILOG 3 (Mislevy & Bock,…

Allowing students to choose the question(s) that they will answer from among several possible alternatives is often viewed as a mechanism for increasing fairness in certain types of assessments. The fairness of optional topic choice is not a universally accepted fact, however, and various studies have been done to assess this question. We examine…

In this paper I describe and illustrate the Roussos-Stout (1996) multidimensionality-based DIF analysis paradigm, with emphasis on its implication for the selection of a matching and studied subtest for DIF analyses. Standard DIF practice encourages an exploratory search for matching subtest items based on purely statistical criteria, such as a…

The effect of item parameters (discrimination, difficulty, and level of guessing) on the item-fit statistic was investigated using simulated dichotomous data. Nine tests were simulated using 1,000 persons, 50 items, three levels of item discrimination, three levels of item difficulty, and three levels of guessing. The item fit was estimated using…

Test validity issues considered by test developers and school districts rarely include individual item readability levels. In this study, items from a major standardized test were examined for individual item readability level and item difficulty. The Homan-Hewitt Readability Formula was applied to items across three grade levels. Results of…

This article investigates the transition to memory-based performance that commonly occurs with practice on tasks that initially require use of a multistep algorithm. In an alphabet arithmetic task, item response times exhibited pronounced step-function decreases after moderate practice that were uniquely predicted by T. C. Rickard's (1997)…

Newtonian mechanics is a core module in technology courses, but is difficult for many students to learn. Computerized tutoring can assist the teachers to provide individualized instruction. This article presents the application of decision theory to develop a tutoring system, "iTutor", to select optimal tutoring actions under uncertainty of…