A computerized adaptive test (CAT) may be modeled as a closed-loop system, where item selection is influenced by trait level ([theta]) estimation and vice versa. When discrepancies exist between an examinee's estimated and true [theta] levels, nonoptimal item selection is a likely result. Nevertheless, examinee response behavior consistent with…

Introducing students to measurement concepts poses unique instructional challenges; yet, the importance of measurement issues for all data collection scenarios (e.g., the classroom, individual clients, sporting events) requires that students understand the value of basic measurement procedures. Illustrating the basic measurement topics of…

This paper reports the results of three studies designed to test the nomological validity of a consumer well-being (CWB) measure in relation to personal transportation. The CWB measure was developed guided by the theoretical notion that the CWB in relation to personal transportation vehicles is significantly enhanced when the consumption of the…

This study presents the random-effects rating scale model (RE-RSM) which takes into account randomness in the thresholds over persons by treating them as random-effects and adding a random variable for each threshold in the rating scale model (RSM) (Andrich, 1978). The RE-RSM turns out to be a special case of the multidimensional random…

We proposed a higher order latent construct of parenting young children, parenting quality. This higher-order latent construct comprises five component constructs: demographic protection, psychological distress, psychosocial maturity, moral and cognitive reflectivity, and parenting attitudes and beliefs. We evaluated this model with data provided…

A cognitive lab technique (n=156) was used to investigate interactions between individual factors and item factors presumed to affect assessment validity for diverse students, including English language learners. Findings support the concept of "access"--an interaction between specific construct-irrelevant item features and individual…

Rates of accurate recall of the Strong Interest Inventory (SII; L. W. Harmon, J. C. Hansen, F. H. Borgen, & A. L. Hammer, 1994) profile information varied with the amount of time elapsed since the interpretation, the type of SII scale, and whether immediate recall was elicited, but rates did not vary with the strategy used to provide the…

This paper introduces criterion-referenced tests (CRTs), compares them with norm-referenced tests (NRTs), discusses how they can be evaluated and revised, and presents a study of an actual class and textbook test evaluation using CRTs. NRTs have dominated testing methodology since the mid-1970s; an example is the Test of English as a Foreign…

A study was designed to provide recommendations regarding the use of the achievement levels set in 1992 for reporting National Assessment of Educational Progress (NAEP) reading results in 1994 and in future NAEP reading assessments. Two procedures were used: the Item Difficulty Categorization (IDC) procedure involved an evaluation of the…

This paper presents a technique for applying the Rule Space Model of cognitive diagnosis (Tatsuoka, 1983) to assessment in a semantically rich domain. Responses of 122 architects to 22 architecture test items developed to assess a range of architectural knowledge were analyzed using Rule Space. Verbal protocol analysis guided the construction of a…

This technical report describes the development of the College-Level Academic Skills Test (CLAST), an instrument designed to measure Florida college students' achievement of the computation and communication skills expected by the completion of their sophomore year. Section I covers CLAST's background and purpose, the requirement that all students…

A study was undertaken to identify the prerequisite relations (or hierarchies among the items) existing in the item responses of a sample of 86 foreign students who took the Test of English as a Foreign Language (TOEFL) vocabulary and reading comprehension test, Form 3JTF1. The form contains 30 vocabulary items and 30 reading comprehension items.…

In item response theory (IRT), preequating depends upon item parameter estimate invariance. Three separate simulations, all using the unidimensional three-parameter logistic item response model, were conducted to study the impact of the following variables on preequating: (1) mean differences in ability; (2) multidimensionality in the data; and…

The purpose of this study was to determine whether item difficulty is significantly affected by language difficulty and response set convergence. Language difficulty was varied by increasing sentence (stem) length, increasing syntactic complexity, and substituting uncommon words for more familiar terms in the item stem. Item wording ranged from…

Four methods for determining the dimensionality of a set of test items were compared: (1) linear factor analysis; (2) residual analysis; (3) nonlinear factor analysis; and (4) Bejar's method. Five artificial test data sets (for 40 items and 1500 examinees) were generated, consistent with the three-parameter logistic model and the assumption of…