Quality assurance and evaluation research, like other fields of social research and its application, are confronted with a series of problems. In the present paper, I want first to give a list of such problems, although necessarily incomplete. It is then claimed that while there is no "perfect" solution to these problems, critical multiplism may…

The No Child Left Behind (NCLB) Act of 2001 requires that public schools adopt research-supported programs and practices, with a strong recommendation for randomized controlled trials (RCTs) as the "gold standard" for scientific rigor in empirical research. Within that policy framework, this paper compares the relative utility of…

The growth of educational teams in collaborative reform initiatives spurs a need for team assessments to assist in promoting functional team environments, productive team work, and member satisfaction. This study applied a multitrait-multimethod framework (D. Campbell and D. Fiske, 1959) to assess team functioning and team leadership. Four case…

Based on the assumption that changes in the educational experience require changes in the evaluation of student learning, this fourth guidebook in a series of nine video conferences on school restructuring presents information on the evolution of student assessment. Four shifts in educational assessment are identified: from testing to multiple…

There is widespread concern about the use and abuse of alcohol and other drugs by adolescents and young adults. It has yet to be determined whether drug use in itself is invariably associated with disruptive or problem use of drugs. Additionally, drug use may vary by frequency and quantity of ingestion which may be differentially related to…

Items from two North American instruments, designed to measure students' evaluation of teaching effectiveness, were administered to university students in different countries. Most of the items were judged by the students to be appropriate and important. Statistical analysis showed strong support for both the convergent and divergent validity of…

In evaluation, the whole is greater than the sum of the parts when quantitative and qualitative approaches and methods are combined. Some circumstances in which both approaches can be effectively combined are considered, and case studies of two evaluations using multiple methods are presented. (LMO)

Whether 12 university students could discriminate validly between the correctness and originality of responses of 150 elementary school children to 3 creativity tests was studied as a determination of the discriminant validity of the tests through the multitrait-multimethod procedure. There was compelling evidence for convergent and discriminant…

Examined the usefulness of a multiple-stage screening procedure in identifying youth experiencing elevated levels of anxiety. Children in grades 4-7 completed a 3-stage screening procedure. Results paralleled outcomes for when the procedure was used to recognize depressed children. Concludes that multiple-stage screening can identify children…

Conflicting evidence regarding listening comprehension as a separate trait in language proficiency is reviewed. Two studies conducted in Japan are presented that use the multitrait-multimethod methodology to examine the construct validity of the listening trait. (74 references) (Author/LB)

Used a multitrait-multimethod approach to investigate the validity of scores on locally prepared performance assessments taken by 6,000 third and fourth graders in reading, language arts, and mathematics. A partial explanation for the lack of construct validity evidence for the performance assessment scores is the probable low reliability of…

In this article we present a cognitive-ecological model for understanding and preventing emotional and behavioral difficulties and propose directions for school-based intervention programs, particularly with aggressive children. In the cognitive-ecological framework, intervention efforts should target certain cognitive skills (e.g., skills that…

Several multivariate statistical methodologies have been proposed to ensure objective and quantitative evaluation of the multitrait-multimethod matrix. The paper examines the performance of confirmatory factor analysis and covariance component models. It is shown, both empirically and formally, that confirmatory factor analysis is not a reliable…

The relationship of multiple linear regression to various multivariate statistical techniques is discussed. The importance of the standardized partial regression coefficient (beta weight) in multiple linear regression as it is applied in path, factor, LISREL, and discriminant analyses is emphasized. The multivariate methods discussed in this paper…

Assessed the convergent and discriminant validity of various measures of family cohesion and control, using a multitrait, multimethod approach. Families with at least one adolescent and in family therapy completed self-report measures and engaged in Reiss' Card Sort Procedure. Ratings of the families' cohesiveness were also obtained from the…