A study of 82 preschoolers determined that a new play assessment (Child-Initiated Pretend Play Assessment), which identifies cognitive play skills, possessed acceptable interrater reliability and could discriminate between the play of typically developing preschoolers and those with preacademic problems. (Contains 65 references.) (JOW)

This study describes the four-phase process used in developing the "Employer Openness Survey" (EOS). The EOS is an 18-item instrument designed to measure the openness of employers to hiring, accommodating, and promoting workers with disabilities. During the first phase, the authors generated potential questions and pilot-tested them with…

The issue of arriving at agreement over outcomes in summative assessment of portfolios has been a major concern, given the complexity of the assessment task, the educational and political context, and the widespread and growing use of portfolios in higher education. This article examines research findings in this area. The discussion takes place…

When raters assign different scores to a performance task, a method for resolving rating differences is required to report a single score to the examinee. Recent studies indicate that decisions about examinees, such as pass/fail decisions, differ across resolution methods. Previous studies also investigated the interrater reliability of…

This article describes the development of assessment criteria for specific aspects of examination answers and coursework essays in psychology. The criteria specified the standards expected for seven aspects of students' work: addressing the question, covering the area, understanding the material, evaluating the material, developing arguments,…

Many studies have indicated that at least 2 raters should score writing assessments to improve interrater reliability. However, even for assessments that characteristically demonstrate high levels of rater agreement, 2 raters of the same essay can occasionally report different, or discrepant, scores. If a single score, typically referred to as an…

Two studies assessed the reliability and utility of the Stimulus Preference Coding System (SPCS) to measure approach, avoidance, and happy and unhappy behaviors in persons with developmental disorders. Study 1 took place in an institutional setting. The nine participants were all adults with mental retardation and multiple associated disabilities.…

Play assessment is gaining attention as a measure of the developing skills of young children. The procedures and methods of coding child behaviours vary considerably across researchers and practitioners. Because of this, definitive statements about the use of play assessment cannot be made without further research. The present study is an attempt…

Objective: Given the present emphasis on accountability and maintaining quality, the objective of this study was to develop, apply, and assess the reliability of a fidelity rating instrument for consumer-operated services--a promising model, but one for which fidelity criteria are not yet established. Method: Based on observations, documents, and…

The piece that follows had its first hearing as part of a symposium at the 1990 annual meeting of the American Educational Research Association in Boston. Sigrun Gudmundsdottir and Grace Grant were at the podium, presenting their two narrative accounts of one schoolteacher's practice. The schoolteacher was present as well, through the medium of a…

The application range of Cohen's Kappa is extended to the field of sequential observation data, where omission mistakes of an observer may often occur. It is shown how the omission mistakes can be incorporated into the calculation of the Kappa coefficient without violating the statistic it is based on. The enhanced coefficient is termed Kappa…

This study examined the impact of treatment adherence and therapist competence on treatment outcome in a controlled trial of individual cognitive-behavioral therapy (CBT) and multidimensional family therapy (MDFT) for adolescent substance use and related behavior problems. Participants included 136 adolescents (62 CBT, 74 MDFT) assessed at intake,…

The training of raters for writing assessment through web-based programmes is emerging as an attractive and flexible alternative to the conventional method of face-to-face training sessions. Although some online training programmes have been developed, there is little published research on them. The current study aims to compare the effectiveness…

Purpose: The purpose of this article is to highlight the results of the online Delphi research project; in particular the procedures used to establish an online and innovative process of content validation and obtaining "rich" and descriptive information using the internet and current e-learning technologies. The online Delphi was proven to be an…

This study examined comparability of student scores obtained from computerized and paper-and-pencil formats of the large-scale statewide end-of-course (EOC) examinations in the two subject areas of Algebra and Biology. Evidence in support of comparability of computerized and paper-based tests was sought by examining scale scores, item parameter…