Student learning is a complex incremental cognitive process; assessment needs to parallel this, reporting the results in similar terms. Application of fuzzy sets and logic to the criterion-referenced assessment of student learning is considered here. The constrained qualitative assessment (CQA) system was designed, and then applied in assessing a…

This study examined the construct validity of the "e-rater"® automated essay scoring engine as an alternative to human scoring in the context of TOEFL® essay writing. Analyses were based on a sample of students who repeated the TOEFL within a short time period. Two "e-rater" scores were investigated in this study, the first…

Background: The ComFor (Forerunners in Communication) is an instrument to explore underlying competence for augmentative communication. More specifically, it measures perception and sense-making of non-transient forms of communication at the levels of presentation and representation. The target group consists primarily of individuals with autism…

Recent estimates of Shaken Baby Syndrome awareness suggest that approximately half of all American adults have not heard of the often devastating risks of shaking an infant. Using a sample of 288 undergraduate students, we developed a measure of attitudes around infant care practices. A total of 264 community participants completed a revised…

The relationship between levels of reliability and the power of two bias and differential item functioning (DIF) detection methods is examined. Both methods, the Mantel-Haenszel (MH) procedure of P. W. Holland and D. T. Thayer (1988) and the Simultaneous Item Bias (SIB) procedure of R. Shealy and W. Stout (1991), use examinees' raw scores as a…

A preliminary study is reported of the use of new multifaceted Rasch measurement mechanisms for investigating rater characteristics in language testing. Ratings from four judges of scripts from 50 candidates taking the International English Language Testing System test, a test of English for Academic Purposes, are analyzed. The analysis…

This paper explores the comparability of item calibrations for three types of items: (1) text only; (2) text with photographs; and (3) text plus graphics when items are presented on written tests and computerized adaptive tests. Data are from five different medical technology certification examinations administered nationwide in 1993. The Rasch…

To determine the most accurate group-administered measure of reading achievement, a study explored variations of the cloze and maze procedures with second grade students who were native English speakers or who were being taught English as a second language. Subjects--108 second grade volunteers (1% American Indian, 49% Asian, 39.8% Black, 1%…

A study assessed the reliability and some aspects related to the validity of an open sentence approach to assessing comprehension during silent reading. Subjects, 37 randomly selected seventh-grade students, read an "Opin" passage, prepared by deleting the second half of every other sentence. The blanks filled in by students were…

A comparison was made of Durrell Analysis of Reading Difficulty and informal reading inventory (IRI) independent and instructional level designations for 15 children, aged 9 to 11 years, who had been referred to a diagnostic clinic for reading assessment. The children's reading performance was first scored according to procedures outlined in the…

This report describes a study comparing the classification results obtained from a one-parameter and three-parameter logistic based tailored testing procedure used in conjunction with Wald's sequential probability ratio test (SPRT). Eighty-eight college students were classified into four grade categories using achievement test results obtained…

Procedures for determining cutting scores have been proposed by Angoff and by Nedelsky. Nedelsky's approach requires that a rater examine each distractor within a test item to determine the probability of a minimally competent examinee answering correctly; whereas Angoff uses a judgment based on the whole item, rather than each of its components.…

The precision and efficiency of a cognitive test constructed by three different methods of item analysis was compared, using the verbal aptitude subtest of the Florida Twelfth Grade Test. Classical item analysis, factor analysis and the Rasch logistic model were used in the construction of 15 and 30 item subtests and replicated for samples of 250,…

Since the accuracy and validity of occupational data may vary according to the rating scale format employed, the first phase of the research described in the report employed hypothetical job descriptions from which accurate criterion data could be generated. The second phase of the research required developing an occupational survey instrument…

The purpose of this study was to explore the usefulness of the Rasch test analysis model for providing consistent scale statistics between content-referenced and norm-referenced tests across school districts. A number of content-referenced tests were created to assess student performance relative to basic skills goals identified in reading and…