When assessment tasks are set for students in universities and colleges, a common practice is to advise them of the criteria that will be used for grading their responses. Various schemes for using multiple criteria have been widely advocated in the literature. Each scheme is designed to offer clear benefits for students. Breaking down holistic…

Two studies extended psychometric research on the Student-Teacher Relationship Scale (STRS) with kindergarten and preschool children (N[subscript 1] = 60-7[subscript 1]; N[subscript 2] = 35) and their teachers. These studies used a multi-method approach to replicate and extend previous findings concerning the convergent validity of the STRS…

This article examines the psychometric structure of the Anxiety Control Questionnaire (ACQ) in Chinese adolescents. With the data collected from 212 senior high school students (94 females, 110 males, 8 unknown), seven models are tested using confirmatory factor analyses in the framework of the multitrait-multimethod strategy. Results indicate…

Combining diverse methods in a single study raises a problem: What should be done when the findings of one method of investigation conflict with those of another? We illustrate this problem using an example in which three study phases--quantitative, qualitative, and intervention--were applied. The findings coming from the quantitative phase did…

Existing research posits multiple dimensions of bullying and victimization but has not identified well-differentiated facets of these constructs that meet standards of good measurement: goodness of fit, measurement invariance, lack of differential item functioning, and well-differentiated factors that are not so highly correlated as to detract…

Approaches to the evaluation of multitrait-multimethod data in which there are multiple indicators of each trait-method combination are explored. A first-order factor defined by multiple indicators is posited for each trait-method combination, and method and trait factors are posited as second-order factors. (SLD)

A study was conducted comparing the feedback received from students about teaching obtained using different instruments. Twelve first- and second-year undergraduate modules were selected from seven different schools within a single university. Students studying each module were allocated to "questionnaire" and "comparator" groups. "Questionnaire"…

The criteria for assessing convergent and discriminant validity proposed by D. T. Campbell and D. W. Fiske (1959) are shown to be inadequate for either the additive or multiplicative structures of data in a multitrait-multimethod matrix. Model-specific criteria are more promising for assessing convergent and discriminant validity. (Author/SLD)

The purpose of this study was to compare multiple regression with canonical analysis in order to introduce a new, policy relevant methodology to the literature on production functions. Findings from this study confirmed the results of past inquiries that found a relationship between the inputs to schooling and measures of student achievement. A…

Multiple evaluation methods (MEMs) can cause an inferential challenge, although there are strategies to strengthen inferences. Practical and theoretical issues involved in the use by social scientists of MEMs, three potential problems in drawing inferences from MEMs, and short- and long-term strategies for alleviating these problems are outlined.…

Background: A programme of construct validity research is necessary to clarify previous research on self-regulation and to provide a stronger basis for future research. Aim: A multitrait-multimethod (MTMM) analysis was conducted to assess convergent and discriminant validity of three self-regulation measures: the Learning and Study Strategies…

A mixed-method conceptual framework was developed from the literature and refined through an analysis of 57 empirical mixed-method evaluations. Five purposes for mixed-method evaluation are identified (triangulation, complementarity, development, initiation, and expansion) and analyzed using the framework of 7 design characteristics. (TJH)

Data from four studies were used to analyze the hypothesis that performance evaluation variables can be constructed with a rater's facet and a trait's facet. The regularity of the pattern of intercorrelations across different rater groups was remarkable. The use of facet analysis before confirmatory analysis is discussed. (SLD)

The evaluation of an intervention delivered across multiple treatment sites presents a unique opportunity for evaluators to gauge the manner and degree to which the "impact" of treatment varies across implementation conditions and different target populations. However, the availability of implementation data for each treatment site, while…

In this article we examine empirical evidence on the criterion, predictive, transfer, and fairness aspects of validity of a large-scale language arts performance assessment, referred to as the Performance Assignment (PA). We use multilevel models to avoid biased inferences that might result from the naturally nested data. Specifically, we examine…