Weighting responses to Constructed-Response (CR) items has been proposed as a way to increase the contribution these items make to the test score when there is insufficient testing time to administer additional CR items. The effect of various types of weighting items of an IRT-based mixed-format writing examination was investigated.…

Traditionally, error in equating observed scores on two versions of a test is defined as the difference between the transformations that equate the quantiles of their distributions in the sample and population of test takers. But it is argued that if the goal of equating is to adjust the scores of test takers on one version of the test to make…

This paper uses the data from a survey among school teachers to conduct a series of factor analysis to test the reliability of a set of items to determine the factors deemed important in technology integration among teachers. The results suggest that there are specific dimensions of items that can be used to determine the factors perceived by…

A new test assembly algorithm based on a Monte Carlo random search is presented in this article. A major advantage of the Monte Carlo test assembly over other approaches (integer programming or enumerative heuristics) is that it performs a uniform sampling from the item pool, which provides every feasible item combination (test) with an equal…

A method for incorporating maximum likelihood (ML) estimation into reliability analyses with item-level missing data is outlined. An ML estimate of the covariance matrix is first obtained using the expectation maximization (EM) algorithm, and coefficient alpha is subsequently computed using standard formulae. A simulation study demonstrated that…

This study investigated the statistical and practical significance of context effects for items randomized within testlets for administration during a series of computerized non-adaptive tests. One hundred and twenty-five items from four primary school reading tests were studied. Logistic regression analyses identified from one to four items for…

Ipsative data (individual scores subject to a constant-sum constraint), suggested to minimize response bias, are sometimes observed in behavioral sciences. Chan and Bentler (1993, 1996) proposed a method to analyze ipsative data in a single-group case. Cheung and Chan (2002) extended the method to multiple-group analysis. However, these methods…

Educational tests used for accountability purposes must represent the content domains they purport to measure. When such tests are used to monitor progress over time, the consistency of the test content across years is important for ensuring that observed changes in test scores are due to student achievement rather than to changes in what the test…

The Common European Framework of Reference (CEFR) is intended as a reference document for language education including assessment. This article describes a project that investigated whether the CEFR can help test developers construct reading and listening tests based on CEFR levels. If the CEFR scales together with the detailed description of…

Trait psychology implicitly assumes consistency of the personal traits. Mischel, however, argued against the idea of a general consistency of human beings. The present article aims to design a statistical procedure based on an adaptation of the pi* statistic to measure the degree of intraindividual consistency independently of the measure used.…

This article illustrates the application of the Echo approach, originally designed to identify values of different cultures and subcultures, to the generation of questionnaire items for students to evaluate faculty teaching performance. Students preferred items generated using the Echo method over faculty-designed items and items developed by…

The authors present a Markov Chain Monte Carlo (MCMC) parameter estimation procedure for the generalized graded unfolding model (GGUM) and compare it to the marginal maximum likelihood (MML) approach implemented in the GGUM2000 computer program, using simulated and real personality data. In the simulation study, test length, number of response…

Textbooks and websites today abound with real data. One neglected issue is that statistical investigations often require a good deal of "cleaning" to ready data for analysis. The purpose of this dataset and exercise is to teach students to use exploratory tools to identify erroneous observations. This article discusses the merits of such…

This study explored the differential performances of mathematics test items used to test secondary school girls and boys in the national examination. The main purpose was to find out whether type of items is the reason for girls' overachievement in the Malaysian mathematics national examination. To investigate seven types of items, Differential…

Modern scale construction techniques have been used to develop scales measuring examiner ratings of examinees' test session behavior (TSB) on Wechsler and Stanford-Binet intelligence tests. This study analyzes data from the Test Session Observation Checklist (TSOC), a measure developed by post hoc rational analysis, from a portion of the Woodcock…