Traditionally, researchers employ human raters for scoring responses to creative thinking tasks. Apart from the associated costs this approach entails two potential risks. First, human raters can be subjective in their scoring behavior (inter-rater-variance). Second, individual raters are prone to inconsistent scoring patterns…

Purpose: Anomia, or word-finding difficulty, is a prevalent and persistent feature of aphasia, a neurogenic language disorder affecting millions of people in the United States. Anomia assessments are essential for measuring performance and monitoring outcomes in clinical settings. This study aims to evaluate the reliability of response time (RT)…

Effective personalization of education requires knowing how each student will perform under certain conditions, given their specific characteristics. Thus, the demand for interpretable and precise estimation of heterogeneous treatment effects is ever-present. This paper outlines a new approach to this problem based on the Leave-One-Out Potential…

This study evaluated the forecasting accuracy of trend estimation methods applied to time-series data from computer adaptive tests (CATs). Data were collected roughly once a month over the course of a school year. We evaluated the forecasting accuracy of two regression-based growth estimation methods (ordinary least squares and Theil-Sen). The…

Purpose: This study aimed to use enrollment information including demographic, family background and financial status, which can be gathered before the first semester starts, to construct early prediction models (EPMs) and extract crucial factors associated with first-year student dropout probability. Design/methodology/approach: The real-world…

The purpose of this study was to examine the role of achievement goals in online test anxiety and help-seeking while controlling for self-efficacy and potential demographic differences. A total of 150 online students participated in the survey. Separate hierarchical regression analysis results suggested the differential roles of achievement goals…

A difficult result to interpret in Computerized Adaptive Tests (CATs) occurs when an ability estimate initially drops and then ascends continuously until the test ends, suggesting that the true ability may be higher than implied by the final estimate. This study explains why this asymmetry occurs and shows that early mistakes by high-ability…

Disproportionately large numbers of Black youth experience repeated encounters with the juvenile justice system, and often these same youth present with a variety of school-related difficulties. Data from a sample of 842 Black males, assessed in five Ohio juvenile courts using the Global Risk Assessment Device (GRAD), are used to describe the…

This paper reports analysis of 2006-2007 on-entry assessment data from the Performance Indicators in Primary Schools Baseline Assessment (PIPS-BLA) of random samples of students in England, Scotland, New Zealand and Australia. The analysis aimed, first, to investigate the validity and reliability of that instrument across countries and sexes, and,…

Today's college ACCUPLACER System's Computerized Placement Test (CPT) plays a major role for determining whether post-secondary students are ready for placement into Arithmetic, Elementary Algebra, Intermediate Algebra, or College Mathematics courses. It is widely used in most college systems in the U.S. This paper discusses some of the issues to…

The utility of computer analysis in the assessment of written products was studied using the WordMAP software package. Data were collected for 92 college freshmen, using: (1) the Test of Standard Written English (TSWE); (2) the English Composition Test of the College Board; (3) verbal and mathematical Scholastic Aptitude Tests; (4) two narrative…

Using simulated item response data, the performance of several "robust" and conventional schemes for ability estimation was evaluated in conjunction with logistic item response theory models (one, two, and three parameter models). The simulated item response data were generated using a model that is more complex than are the usual…

The relationship between selected subtests from the Armed Services Vocational Aptitude Battery (ASVAB) and corresponding subtests administered as computerized adaptive tests (CAT) was investigated using a sample of Marine recruits. Results showed that the CAT subtest scores correlated as well with initial ASVAB scores as did ASVAB retest scores,…

This paper reports the results of using a three-layer backpropagation artificial neural network to predict item difficulty in a reading comprehension test. Two network structures were developed, one with and one without a sigmoid function in the output processing unit. The data set, which consisted of a table of coded test items and corresponding…

A nominal response model-based computerized adaptive testing procedure (nominal CAT) was implemented using simulated data. Ability estimates from the nominal CAT were compared to those from a CAT based upon the three-parameter logistic model (3PL CAT). Furthermore, estimates from both CAT procedures were compared with the known true abilities used…