This research aimed to investigate the effect of computers on the test and inter-rater reliability of writing test scores of ESL learners. Writing samples of 20 pen-paper and 20 computer group students were scored in analytic scoring method by two scorers, and then the scores were analyzed in Alpha (Cronbach) model. The results showed that the…

A total of 2,000 essays written by 1,000 students was submitted to generalizability analyses for domain-referenced tests. Each student had written one essay on each of two prompts representing two models of discourse. Each essay was read by six readers and judged on a scale of from 1 to 4. No reader read essays from both prompts. Reader agreement…

This study examined whether variations in the Developmental Observation Checklist (DC) format influences congruence of scores among both parents and the child's teacher. The DC was varied by adding pictorial illustrations and examples and having three response categories instead of two. Results from 100 sets of participants were evaluated with…

The Child Behavior Checklist and related forms were completed for 171 boys referred for school-based assessment resulting from academic and/or behavioral problems. Adolescents consistently underreported behavioral problems relative to parents and teachers regardless of subsequent diagnosis. Implications of these discrepancies in school-based…

To assess cross-observer agreement on personality profiles, an Index of Profile Agreement and an associated coefficient are proposed that take into account both the difference between the ratings and the extremes of their mean. Data from the Revised NEO Personality Inventory for 250 peer ratings/self-reports and 68 spouse ratings/self-reports…

Investigated the reliability and item appropriateness, as discerned by adults affiliated with an infant center, of the Caregiver-Environment Scale (CES). Found the CES to be an easy to use, reliable instrument for evaluation. (Author/SD)

This study explored the use of the Formal Elements Art Therapy Scale (FEATS) with a population of persons with a DSM-IV diagnosis of Substance Use Disorder who were court ordered for treatment. Two groups of adults (N = 40) were closely matched on age, gender, race, socioeconomic status and education level, and were administered the Person Picking…

The various methods for computing the reliability of scores on Advanced Placement (AP) examinations are summarized. For the free response portion of the examinations, raters can contribute to score unreliability through both systematic severity errors (in which some raters consistently rate more severely than other raters) and through…

This paper argues that special procedures for constructing assessment tools containing performance assessment tasks are unnecessary and that current test methodology can easily be generalized to complex performance assessment tasks without destroying the desirable characteristics of those tasks. Reasonable statistical requirements for sound…

The purpose of this paper is to compare the precision of direct and indirect measures of writing assessment using the test information functions from a graded response Item Response Theory (IRT) model. Subjects were 192 sophomore English students from a parochial high school in Wisconsin. Both direct and indirect measures of writing ability were…

Recently, a great deal of attention has been focused on the development of a system of relational diagnosis to be incorporated into the American Psychiatric Association's diagnostic system, that of the Diagnostic and Statistical Manual (DSM). One of the more intriguing components of this effort is the Global Assessment of Relational Functioning…

This study, involving five experienced and trained speech language pathologists, categorized the elicited imitations of five profoundly and five severely prelingually hearing-impaired subjects using the Phonetic Level Evaluation. Failure to obtain acceptably high levels of reliability suggests that this measure may not yet be an accurate and…

Forty prototypical descriptions representing 4 aptitude domains and 4 talent fields were rated by 2,343 intermediate-level pupils and their teachers, and indices of interpeer agreement were computed. A majority of the prototypes maintained acceptable interpeer agreement levels. Interpeer agreement depended primarily on the specific aptitude or…

To assess the psychometric characteristics of the Verbal Behavior Assessment Scale, the 15-item questionnaire was administered to pairs of caregivers of 115 individuals with developmental disabilities. Exploratory factor analysis involving 11 more participants revealed evidence concerning the distinction of three different communicative functions…

The objectives of this study were: (1) to examine the psychometric properties of the Assessment of Life Habits (LIFE-H) for children; and (2) to draw a profile of the level of participation among children of 5 to 13 years of age with various impairments. The research team adapted the adult version of the LIFE-H in order to render it more…