This instructional module presented by the National Council on Measurement in Education (NCME) provides guidelines for teachers and other test developers to help them construct test item banks. Setting up an item bank and using it are described, with a consideration of software that can be used. (SLD)

Effects of self-adapted testing (SAT), in which examinees choose the difficulty of items themselves, on ability estimates, precision, and efficiency, mechanisms of SAT effects, and examinee reactions to SAT are reviewed. SAT, which is less efficient than computer-adapted testing, is more efficient than fixed-item testing. (SLD)

Several existing group-based statistics to detect improbable item score patterns are discussed, along with the cut scores proposed in the literature to classify an item score pattern as aberrant. A simulation study and an empirical study are used to compare the statistics and their use and to investigate the practical use of cut scores. (SLD)

Explores recent developments in the use of computers in language testing in four areas: (1) item banking; (2) computer-assisted language testing; (3) computerized-adaptive language testing; and (4) research on the effectiveness of computers in language testing. Examines educational measurement literature in an attempt to forecast the directions…

Studied the effects of item content and stereotypic threat to gender differences in mathematical problem solving through two experiments, one with 63 seventh and eighth graders and the other with 174 college students. Results suggest that gender-stereotype threat could be a big factor in gender differences in mathematical problem solving. (SLD)

Examiners seeking guidance on multiple-choice and true/false tests are likely to encounter various faulty or questionable ideas. Twelve of these are discussed in detail, having to do mainly with the effects on test reliability of test length, guessing and scoring method (i.e. number-right scoring or negative marking). Some misunderstandings could…

In the Programme for International Student Assessment (PISA) the items are organised in small clusters relating to the same stimulus material (called 'units'). Homogeneity analysis (HA) is used to develop a detailed description of the relationship between all the items in one unit, using the categorical information available in the PISA data. The…

In this paper we have set out to search for similarities and differences between the Nordic countries concerning patterns of competencies defined as scientific literacy in the Programme for International Student Assessment (PISA) study. The first part focuses on gender differences concerning the two types of competencies, understanding of…

Weighting responses to Constructed-Response (CR) items has been proposed as a way to increase the contribution these items make to the test score when there is insufficient testing time to administer additional CR items. The effect of various types of weighting items of an IRT-based mixed-format writing examination was investigated.…

This paper uses the data from a survey among school teachers to conduct a series of factor analysis to test the reliability of a set of items to determine the factors deemed important in technology integration among teachers. The results suggest that there are specific dimensions of items that can be used to determine the factors perceived by…

A new test assembly algorithm based on a Monte Carlo random search is presented in this article. A major advantage of the Monte Carlo test assembly over other approaches (integer programming or enumerative heuristics) is that it performs a uniform sampling from the item pool, which provides every feasible item combination (test) with an equal…

A method for incorporating maximum likelihood (ML) estimation into reliability analyses with item-level missing data is outlined. An ML estimate of the covariance matrix is first obtained using the expectation maximization (EM) algorithm, and coefficient alpha is subsequently computed using standard formulae. A simulation study demonstrated that…

This study investigated the statistical and practical significance of context effects for items randomized within testlets for administration during a series of computerized non-adaptive tests. One hundred and twenty-five items from four primary school reading tests were studied. Logistic regression analyses identified from one to four items for…

Ipsative data (individual scores subject to a constant-sum constraint), suggested to minimize response bias, are sometimes observed in behavioral sciences. Chan and Bentler (1993, 1996) proposed a method to analyze ipsative data in a single-group case. Cheung and Chan (2002) extended the method to multiple-group analysis. However, these methods…

Educational tests used for accountability purposes must represent the content domains they purport to measure. When such tests are used to monitor progress over time, the consistency of the test content across years is important for ensuring that observed changes in test scores are due to student achievement rather than to changes in what the test…