The reason for conducting the present case study was the problems the researchers encountered during data collection for another research project (Primary Study) entitled "The effects of virtual versus traditional flipped classes on EFL learners' grammar knowledge, self-regulation, and autonomy." Two online questionnaires were…

Assessment rubrics are being promoted and introduced into tertiary teaching practices on the grounds that they are an efficient and reliable tool to evaluate student performance effectively and promote student learning. However, there has been little discussion on the value of using assessment rubrics in higher education. Rather, they are being…

Cost-effectiveness analysis is rarely used in education. When it is used, it often fails to meet methodological standards, especially with regard to cost measurement. Although there are occasional criticisms of these failings, we believe that it is useful to provide a listing of the more common concerns and how they might be addressed. Based upon…

Despite considerable interest in the topic of instructional quality in research as well as practice, little is known about the quality of its assessment. Using generalizability analysis as well as content analysis, the present study investigates how reliably and validly instructional quality is measured by observer ratings. Twelve trained raters…

This experimental project investigated the reliability and validity of rubrics in assessment of students' written responses to a social science "writing prompt". The participants were asked to grade one of the two samples of writing assuming it was written by a graduate student. In fact both samples were prepared by the authors. The…

P. R. Killeen's (2005a) probability of replication ("p[subscript rep]") of an experimental result is the fiducial Bayesian predictive probability of finding a same-sign effect in a replication of an experiment. "p[subscript rep]" is now routinely reported in "Psychological Science" and has also begun to appear in…

This paper forms part of an exploration of assessment on one part-time higher education (HE) course: an in-service, professional qualification for teachers and trainers in the learning and skills sector which is delivered on a franchise basis across a network of further education colleges in the north of England. This paper proposes that the…

Conducted studies using two perspectives for obtaining evidence relevant to validity of autobiographical data. In Study 1, construct validity of standard psychological constructs (assertiveness, trustworthiness) obtained from autobiographies was equal to that of standard instruments designed to assess these constructs. Evidence for validity of…

This paper summarizes the results of a quantitative review of the validity of the employment interview. It begins by discussing factors that make the interpretation of interview validity data difficult. The interpretation of interview validity data is shown to be hampered by the heterogeneity of constructs and measurement methods in the interview.…

Examines the construct and action validity of the use of simple checklists to assess the declarative knowledge component of adult's literacy by telephone. Finds significant relationships among knowledge when assessed either by listening (telephone) or by reading (mailed questionnaire) modality. Concludes that the telephone method may provide a…

Four experienced stuttering researchers viewed videodisks of spontaneous speech from chronic stutterers and attempted to locate the precise onset and offset of individual stuttering events. Results showed interjudge disagreements that challenge the reliability and validity of onset and offset judgments. Highly agreed stuttering events were…

Evaluation research is discussed as an applied social science that uses quantitative-comparative techniques to assess the effects of planned intervention. Focus is on the generally insufficient validity of treatment effectiveness research, and the relationship between research constructs and theory and their operationalization. (TJH)

A year-long case study of the effects of an educational and vocational rehabilitation program in a youth training center was discussed in terms of fourteen issues which might threaten the validity of an evaluation or research project. This correctional institution housed women aged 14 to 21 who were long-term wards of the state, held on remand, or…

A large data set of student ratings of college teaching was studied to explore previous indications that student ratings are affected by the subject area of instruction. Previous research by Marsh (1984), Feldman (1978), and Cranton and Smith (1986) reported the influence of factors other than instruction. To examine the impact of subject area on…

This study examined the magnitude of differences in standard scores, convergent validity, and concurrent validity when an individual's performance was gauged using the revised and the normative update (Woodcock, 1998) editions of the Woodcock Reading Mastery Test in which the actual test items remained identical but norms have been updated. From…