Four approaches to estimating interrater reliability in early childhood special education research are illustrated and compared: correlation, comparison of means, percentage of agreement, and generalizability theory techniques. Generalizability theory techniques are proposed as a method for estimating the amount of variance attributable to…

Describes holistic scoring as one of the biggest breakthroughs in writing assessment. Suggests that the technique's high interrater reliability coefficients partly explain holistic scoring's popularity. Argues that validity has been largely neglected. Concludes that more must be learned about the uses and effects of holistic scoring. (SG)

Education remains heavily shackled by punitive, test-driven reform. Despite reasonable alternatives, testing increasingly drives educational accountability and reform. Standardization of direct writing assessments promotes scoring reliability and facilitates educational comparisons and rankings. However, standardized writing is not good writing,…

A quantitative study modeled the inherently subjective admissions process for 592 graduate school candidates and 72 raters. Logistic regression models were well-fitting and parsimonious, allowing analysis of each stage of the process. Extended committee discussion/deliberation phases were of limited productivity when inter-rater agreement was…

Two experiments examined infants' reliance on object shape versus color for word generalization to animate and inanimate objects. Infants were taught labels for either novel vehicles or novel animals using preferential-looking procedure or an interactive procedure. Results of both experiments indicated that infants limited their word…

This study assessed the interrater reliability of a Swedish version of the Childhood Autism Rating Scale (CARS), an instrument for screening and diagnosis of autism. The CARS was used for rating autistic behavior by two investigators in 25 children. Results indicated fair to excellent agreement. Aspects of validity and reliability are discussed.…

Studied the marking characteristics of experienced markers of GCSE role-play to ascertain the extent of variation between the marks they award and to determine whether more intermarker consistency could be obtained with a small increase in time for thought before a particular mark is awarded. Results underlie the importance of reducing the…

Analyzes simulated performance ratings on a six-point scale by two independent raters to account for nonsystematic error in performance ratings. Results suggest that rater agreement or covariance is not always a dependable estimate of score reliability and that the practice of seeking additional raters for adjudication of discrepant ratings is not…

Ten speech-language pathology students judged five-second audiovisually recorded speech intervals as stuttered or nonstuttered in group and single-subject experiments. Results showed that judgment accuracy tended to increase after training, both for speakers used during the training process and unfamiliar speakers. Slight increases in interjudge…

Examined low interrater agreement by decomposing common and unique variance among parent, adolescent, and observer reports of parental warmth and negativity into genetic and environmental factors. Model-fitting analyses findings generally supported predictions for warmth and negativity at Family and Individual levels. At the Social level, genetic…

The use of anti-epileptic medications (AEDs) is much higher in individuals with intellectual disabilities than in the general population. As many of these individuals rely on such medications, clinicians should consider psychometrically sound instruments for assessing adverse side effects of these medications as one aspect of routine clinical…

Given a growing emphasis on early intervention for children with autism, valid quantitative tools for measuring treatment response are needed. The Social Responsiveness Scale (SRS) is a brief (15-20 minute) quantitative measure of autistic traits in 4-to 18-year-olds, for which a version for 3-year-olds was recently developed. We obtained serial…

In this paper we present initial information concerning a new direct observation system--the Observational System for Recording Physical Activity in Children-Preschool Version. The system will allow researchers to record young children's physical activity levels while also coding the topography of their physical activity, as well as detailed…

Background: The Children's Communication Checklist (CCC 1998) was revised in 2003 (CCC-2) to provide a general screen for communication disorder and to identify pragmatic/social interaction deficits. Two validation studies were conducted with different populations of children with language and communication impairments. Methods & Procedures: In…

People with intellectual disability (ID) and untreated psychiatric disorder lead unnecessarily difficult and unhappy lives. The prevalence of mental illness in children and adults with ID is greater than that found in the general population. A carer-completed checklist of psychopathology that could be used with both children and adults would help…