As evidence-based interventions are scaled, fidelity of implementation, and thus effectiveness, often wanes. Validated fidelity measures can improve researchers' ability to attribute outcomes to the intervention and help practitioners feel more confident in implementing the intervention as intended. We aim to provide a model for the validation of…

In order to construct and evaluate the validity argument for the Objective Structured Assessment of Technical Skills (OSATS), based on Kane's framework, we conducted a systematic review. We searched MEDLINE, EMBASE, CINAHL, PsycINFO, ERIC, Web of Science, Scopus, and selected reference lists through February 2013. Working in duplicate, we selected…

Twenty-two handicapped and 10 nonhandicapped young children were administered the Evaluation and Programing System Level 1 (EPS-1), a measure designed to provide information to develop educational programs and assess program effectiveness. The article notes reliability, validity, and utility data. (Author/CL)

This paper discusses and describes an industry-based process for the design and development of criterion-referenced tests in vocational-technical subject areas to support vocational program evaluations. The following topics are covered in the paper: (1) use of criterion-referenced tests; (2) test design; (3) preparation of test materials; and (4)…

A few multi-dimensional items were found to be more efficient than a larger number of uni-dimensional items in tests for participant selection and evaluation of such compensatory educational programs as Title I. A multi-dimensional scaling technique is described which derives scores for each item on three dimensions, and allows multiple scores to…

Through an extensive nomination process, six school districts which were trying to link their testing and evaluation activities with instruction were identified. Interviews with these districts were conducted over a 2-year period; the results dealing with the dissemination and use of testing for instructional change are presented here. Three…

A method for translating the achievement gains in Normal Curve Equivalents (NCE) to a more conceptually understandable index of academic growth is described. This method calculates "Expected Growth (EG)" and expresses the relationship of observed achievement gains. The method is shown to be easily applied and well suited to local…

The relationships among independent observer ratings of a child care program on the Early Childhood Environment Rating Scale (ECERS), state department personnel ratings of program quality using the Child Development Program Evaluation Scale (CDPES), and self-evaluation ratings using the self-assessment instrument designed for the Early Childhood…

A study sponsored by the Texas Education Agency was conducted to define qualitative and quantitative indicators of school district success. Efforts were focused on: viewpoints of administrators; quality indicators examined in other states; and a review of the literature. Results and recommendations are summarized into the following categories:…

State program practices in norm-referenced testing (NRT) are described as found in the 1990-91 school year and prior years to provide information on years of administration of the same test and the age of the norms. All 50 State Departments of Education were surveyed in 2 stages. States without NRT were asked to described their testing policies;…

The "Survey of Undergraduate Alumni" and the "Survey of Graduate Alumni" for the Department of Health Education at San Francisco State University (California) were developed and evaluated. The surveys were intended to measure alumni satisfaction with the programs in health education. Using a modified Delphi technique, a group…

The ways in which a rating scale developed for problem-based medical curricula provides information that is useful in the area of curriculum reform were demonstrated in a study involving 95 first-year medical students at the University of Sherbrooke in Quebec (Canada). The rating scale, based on the theory of problem-based learning of W. H.…

The perceptions of testing directors, school principals, teacher supervisors, and classroom teachers concerning the actual purposes of public school standardized group testing programs were studied. Surveys were mailed to superintendents in 97 randomly selected public school districts. The superintendents forwarded survey packets to selected…

An evaluation of a school reform project in one kindergarten through grade 5 school in Atlanta (Georgia) is presented to demonstrate how triangulation of qualitative and quantitative evaluation approaches can enhance understanding of a reform effort. During the 1991-92 school year, the final year of the project, Fowler Drive Elementary School…

Arizona teachers and administrators (n=2,444) were surveyed to determine (1) use of standardized test scores; (2) teachers' perceptions of others' use of scores; (3) how schools prepare students to take tests; (4) conditions under which tests are given; and (5) views of current testing program effects. (SLD)