The Association of American Medical Colleges is conducting research to develop, implement, and evaluate a Medical College Admission Test (MCAT) essay testing program. Essay administration in the spring and fall of 1985 and 1986 suggested that additional research was needed on the development of topics which elicit similar skills and meet standard…

This study is an attempt at a cohesive characterization of the concept of essay reliability. As such, it takes as a basic premise that previous and current practices in reporting reliability estimates for essay tests have certain shortcomings. The study provides an analysis of these shortcomings--partly to encourage a fuller understanding of the…

Even though several national testing firms have developed measures to evaluate the effectiveness of baccalaureate education, there continues to be a general reluctance on the part of faculty in colleges and universities to accept these measures as criteria on which to evaluate educational programs. Some of the resistance appears to lie in the lack…

In an assessment of the adequacy of the Gesell screening examination as a test instrument, a Gesell Screening Evaluation was given to 400 children semi-annually from their 4th to 6th year. The sample, which was stratified by parent occupation, included 40 girls and 40 boys at 5 age levels. The test battery corresponded with the Gesell Preschool…

The necessity of controlling the order in which trained essay raters for a statewide writing assessment program receive student essays was studied. The underlying theoretical question concerns possible rater bias caused by raters reading long strings of essays of homogeneous quality; this problem is usually referred to as context effect or…

The purpose of this study was to compare the relative magnitude of the variance components and generalizability coefficients derived from the Teacher Performance Assessment Instruments (TPAI) data using two different methods of data collection: (1) occasions when observers were in the classroom for simultaneous observation and (2) occasions when…

Six university English departments collaborated in this examination of the differences between multiple-choice and essay tests in evaluating writing skills. The study also investigated ways the two tools can complement one another, ways to improve cost effectiveness of essay testing, and ways to integrate assessment and the educational process.…

A study investigated the use of reading proficiency scales developed by the American Council on the Teaching of Foreign Languages (ACTFL), Educational Testing Service (ETS), and Interagency Language Roundtable (ILR) for meaningful rank-ordering and assigning levels of second language competence to reading passages. In a proficiency test writing…

The purposes of this study were to establish the interrater reliability, dimensionality, and internal consistency of an instruction evaluation instrument used at The University of Michigan Medical School. Using the nine-item rating scale, 1,758 student ratings and 88 staff ratings were gathered on 61 faculty. Interrater agreement ranged from .28…

This study addressed seven questions regarding the methods used in setting passing scores on the essay subtest of the National Teacher Examinations (NTE) Communication Skills test for the North Carolina State Board of Education. North Carolina uses these tests to screen prospective applicants to teacher education programs. The judges (five college…

A latent trait method is presented to investigate the possibility that Angoff or Nedelsky judges specify inconsistent probabilities in standard setting techniques for objectives-based instructional programs. It is suggested that judges frequently specify a low probability of success for an easy item but a large probability for a hard item. The…

A study examined teachers' and parents' ratings of 39 developmentally handicapped (DH) and 9 learning disabled children on the revised Vineland and the AAMD Adaptive Behavior Scale. Teachers rated children eligible for DH placement more frequently than parents. The adaptive behavior ratings of the two instruments differed significantly. (Author/CB)

Investigated interrater reliability of the AAMD Adaptive Behavior Scale-Public School Version in a sample of 31 educable mentally handicapped children who were rated by their parents, special education teacher, classroom teacher, and an independent observer. Results showed ratings of the special education teacher were generally lower. (JAC)

A group of seven Japanese-language Oral Proficiency Interview (OPI) testers licensed by the American Council on the Teaching of foreign Languages (ACTFL) conducted research related to ACTFL-OPI criteria. They first examined 24 audiotaped interview tests to see what kind of consistency there would be when individual testers applied general criteria…

The cognitive components standard setting method, recently introduced by D. McGinty and J. Neel (1996), asks judges to specify minimum levels of performance not for the test items, but for smaller portions of items, the component skills and concepts required to answer each item correctly. Items are decomposed into these components before judges…