This paper presents ongoing research at the University of Hertfordshire on the use of computer-adaptive tests (CATs) in Higher Education. A software prototype based on Item Response Theory has been developed and is described here. This application was designed to estimate the level of proficiency in English for those students whose first language…

By the spring of 2000, forty states had begun using student test scores to rate school performance. Twenty states have gone a step further and are attaching explicit monetary rewards or sanctions to a school's test performance. In this paper, the authors focus on accountability programs in which states measure the effectiveness of individual…

In this study, retrospective interviews were used to investigate reliability (and thus validity) threats to a computerized ESL listening comprehension test administered at a university in the US. The participants in the investigation, six international graduate students, were asked to respond to semi- and open-ended questions during individual…

This study evaluates the effects of employing the linear, normalizing, and arcsine transformation methods for constructing scale scores on the Basic Competence Test (BCTEST). Tests in three subject areas (Chinese, English, and Mathematics) were studied using the data of test administrations from 2001 to 2003. The resulting scale scores for each…

This study compared 4,594 student responses from three different surveys of incoming students at the University of South Florida (USF) with data from Florida's State University System (SUS) admissions files to determine what proportion of error occurs in the survey responses. Specifically, the study investigated the amount of measurement error in…

The National Assessment of Educational Progress (NAEP) currently tests seventeen-year-old students enrolled in public and private secondary schools, but it does not test "out-of-school" seventeen-year-olds who have either graduated or dropped out. Estimating that one of five seventeen-year-olds is out of school, the interpretability of…

Linear and equipercentile equating conversions were developed for two forms of the Graduate Record Examinations (GRE) quantitative test and the verbal-plus-quantitative test. From a very large sample of students taking the GRE in October 1981, subpopulations were selected with respect to race, sex, field of study, and level of performance (defined…

This paper explores various procedures of panel data in the estimation of causal models. The reported analyses are from the Monitoring the Future study, a nationwide questionnaire survey of 16,000 to 17,000 high school seniors conducted annually since 1975. First, the parameters of causal models are estimated in which the dependent variables are…

This paper examines how various equating methods are affected by: (1) sampling error; (2) sample characteristics; and (3) characteristics of anchor test items. It reviews empirical studies that investigated the invariance of equating transformations, and it discusses empirical and simulation studies that focus on how the properties of anchor tests…

This study investigated seven methods for analyzing multivariate group differences. Bonferroni t statistics, multivariate analysis of variance (MANOVA) followed by analysis of variance (ANOVA), and five other methods were studied using Monte Carlo methods. Methods were compared with respect to (1) experimentwise error rate; (2) power; (3) number…

A one-parameter latent trait model was the basis of the test development procedures in the Basic Skills Assessment Program (BSAP) of the Department of Defense Dependents Schools (DoDDS). Several issues are involved in applying the Rasch model to an assessment program in a large school district. Separate sets of skills continua are arranged by…

The logic of using a gain score approach versus longitudinal causal models is studied in this secondary analysis of a complex data base. The gain score model used by the Federal Reserve Bank and the School District of Philadelphia in their "What Works in Reading?" study is successively refined using the LISREL structural equation…

The Rasch model for test analysis is described and compared with two-parameter and three-parameter latent-trait models. Conditional maximum likelihood equations for estimating item parameters are derived, and estimates of person parameters are described together with their confidence intervals. Goodness of fit tests are discussed, including a…

Action research often necessitates the use of intact groups for the comparison of educational treatments or programs. This paper considers several analytical methods that might be used for such situations when pretest scores indicate that these intact groups differ significantly initially. The methods considered include gain score analysis of…

Rasch calibration permitted the development of short achievement tests that were economical in testing time, and could be developed in a series of difficulty levels to suit student individual differences. Furthermore, these tests were of adequate reliability for practical educational measurement when individual students were assigned to tests of…