The feasibility of reducing scoring costs for the Test of Spoken English (TSE) by using one rater was investigated. Currently, two raters are used. It was found that, because of the possibility of different standards used by potential raters, it does not appear feasible to use a single rater as the sole determiner of speaking proficiency under the…

Procedures for using the Interactive Teaching Map to collect data and construct reliable measures of program implementation and outcomes are illustrated using data from the Delinquency Research and Development Project. Implementation analysis incorporating measures from classroom observation data can add to traditional experimental control group…

Two research projects focused on use of casual markers (graders) for correcting and grading distance education (DE) students' work. A Charles Sturt University project convened focus groups of DE students, casual DE markers, and lecturers who "managed" markers to uncover concerns. University of South Australia research focused on pedagogical issues…

A holistic approach was used to evaluate student writing skills in a high school that is part of a residential treatment program. Faculty members scored 740 writing samples in the fall and spring of the 1990-91 school year. Good interrater reliability and significant skill gains for students were found. (SLD)

This review of research on second-language oral testing outlines the nature of early research in interview-format proficiency testing, then reports on new directions in investigation of construct validity of interview-format and other oral skills tests through examination of examinee, interviewer, and rater performance. Research on empirically…

The Australian Second Language Proficiency Ratings (ASLPR) is a scale that describes how second language proficiency develops on a scale from zero to native-like proficiency, providing performance descriptions in terms of practical tasks. Initially developed for English second language teaching, it has been adapted for English dialects in…

The statute authorizing the National Assessment of Educational Progress (NAEP) calls for the National Assessment Governing Board (NAGB) to set appropriate achievement levels in all areas and grades tested by the NAEP. These levels are intended to establish what students should know, not just what they do know. In 1990, the NAEP posited three…

The Invitational Teaching Observation Instrument was developed to extend effective teaching through self-assessment and clinical supervision. Based on the theories of Invitational Education, this test analyzed both personal and professional dimensions of teaching. Items reflected research on effective teaching and were cross-validated with two…

In a 6-month study of a small private school (Fielding) in central Virginia, seven participants in a curriculum practicum tested a research strategy for investigating the knowledge, skills, and attitudes fostered by a school's environment. Given the subjectivity of environmental influences, the team rejected scrupulous data collection to…

A method for understanding and controlling the multiple facets of an oral examination (OE) or other judge-intermediated examination is presented and illustrated. This study focused on determining the extent to which the facets model (FM) analysis constructs meaningful variables for each facet of an OE involving protocols, examiners, and…

Technical questions regarding the reliability and validity of large-scale portfolio assessment were studied which focused on: (1) whether raters can score collections of writing reliably with rubrics designed for single samples; (2) whether ratings derived from different frameworks differ in their capacities to support technically sound…

This paper discusses the results of a study that investigated the validity and reliability of the Ecology Rating Scale (ERS). The ERS is a brief, multi-dimensional level-of-functioning instrument that can be rated by parents or clinicians. The ERS is comprised of seven domains of youth functioning: family, school, emotional, legal/justice,…

This study aimed to comprehensively investigate the impact of a word-processor on an ESL writing assessment, covering comparison of inter-rater reliability, the quality of written products, the writing process across different testing occasions using different writing media, and students' perception of a computer-delivered test. Writing samples of…

This packet contains a manual and a workbook for developing performance tests in vocational education. The manual gives an in-depth description of how to develop, score, and use performance tests. It includes the following sections: definitions of performance testing, steps in developing a performance test, selecting a performance development…

Previous literature on classroom testing of second language speech skills provides several models of both task types and rubrics for rating, and suggestions regarding procedures for testing speaking with large numbers of learners. However, there is no clear, widely disseminated consensus in the profession on the appropriate paradigm to guide the…