The purpose of this study was to compare and evaluate five online pretest item calibration/scaling methods in computerized adaptive testing (CAT): (1) the marginal maximum likelihood estimate with one-EM cycle (OEM); (2) the marginal maximum likelihood estimate with multiple EM cycles (MEM); (3) Stocking's Method A (M. Stocking, 1988); (4)…

Reports on a study conducted to evaluate how individual differences in anxiety levels affect performance on computer versus paper-and-pencil forms of verbal sections of the Graduate Record Examination. Contrary to the research hypothesis, analysis of scores revealed that extroverted and less computer anxious subjects scored significantly lower on…

Discusses computer-managed testing and describes a study of elementary school students that compared computerized and paper-and-pencil testing. Guidelines for computerized testing are offered, including decisions relating to hardware requirements, the testing software, and the importance of collaboration between computer programers and test…

Describes a study that was conducted to explore how kindergarten and second-grade students are able to structure and solve geometric analogy problems in a computer-based test and to compare the results to a paper-and-pencil test format. Use of the Test of Analogical Reasoning in Children is described. (18 references) (LRW)

Reviews research literature on comparability issues in computerized adaptive testing (CAT) and synthesizes issues specific to comparability and test security. Develops a framework for evaluating comparability that contains three categories of criteria: (1) validity; (2) psychometric property/reliability; and (3) statistical assumption/test…

In an engineering technology course at Coventry University (England), the utility of computer-assisted tests was compared with that of traditional paper-based tests. It was found that the computer-based technique was acceptable to students, produced valid results, and demonstrated potential for saving staff time. (Author/MSE)

This study has investigated the comparability of computer-based testing writing prompts in the Test of English as a Foreign LanguageTM (TOEFL) for examinees of different native language backgrounds. A total of 81 writing prompts introduced from July 1998 through August 2000 were examined using a 3-step logistic regression procedure for ordinal…

This study assessed the comparability and reliability of paper-based and computer-based administration of psychosocial construct measures related to adolescents engaging in physical activity and sedentary behaviors. Adolescents (n = 76; 55% girls, 49% Hispanic, 18% non-Hispanic-white; M age = 13 years) completed measures of behavior change…

The ever increasing explosion of highly attractive multimedia resources on offer has boosted the use of information and communication technology (ICT) in the teaching and learning of modern languages. The use of ICT to assess languages is less frequent, however, although online testing is starting to develop. This paper examines the national…

In 2004 Embry-Riddle Aeronautical University transitioned from proctored examinations for distance learning courses to online examinations that are not proctored. The purpose of this study was to determine if there is a significant difference in the midterm and final examination grades between proctored and non-proctored online examinations. Three…

The interest in the application of large-scale computerized adaptive testing has served to focus attention on issues that arise when theoretical advances are made operational. Some of these issues stem less from changes in testing conditions and more from changes in testing paradigms. One such issue is that of the order in which questions are…

Two-stage and multistage test designs provide a way of roughly adapting item difficulty to test-taker ability. All test takers take a parallel stage-one test, and, based on their scores, they are routed to tests of different difficulty levels in subsequent stages. These designs provide some of the benefits of standard computerized adaptive testing…

Computerized adaptive testing procedures (CATPs) based on the graded response method (GRM) of F. Samejima (1969) and the partial credit model (PCM) of G. Masters (1982) were developed and compared. Both programs used maximum likelihood estimation of ability, and item selection was conducted on the basis of information. Two simulated data sets, one…

First-, second-, and third-year students enrolled in an introductory educational psychology class at Texas A&M University in College Station were administered either a pencil-and-paper or computerized version of the Strong-Campbell Interest Inventory. The same or other version of the test was administered after 2 weeks. Focus is on equivalence…

Evaluation models are being developed for assessing artificial intelligence (AI) systems in terms of similar performance by groups of people. Natural language understanding and vision systems are the areas of concentration. In simplest terms, the goal is to norm a given natural language system's performance on a sample of people. The specific…