This paper describes, and lays out an argument for, the use of a procedure to help groups of reviewers to judge the quality of prior research reports. It argues why such a procedure is needed, and how other existing approaches are only relevant to some kinds of research, meaning that a review or synthesis cannot successfully combine quality…

Purpose: This paper illustrates how Guba and Lincoln's parallel criteria for establishing trustworthiness, can be adapted and applied to qualitative research on indigenous social protection systems. It provides insights for social protection researchers, exploring plausible qualitative research rigor evaluation criteria, on plausible alternatives.…

This article provides insight into how an adequate approach to selecting methods can establish credible and actionable evidence. The authors offer strategies to effectively support Extension professionals, including program developers and evaluators, in being more deliberate when selecting appropriate qualitative and quantitative methods. In…

Purpose: The purpose of this paper is to present a framework intended to guide students and novice researchers in learning about the necessary dimensions for assessing the rigour of qualitative research studies. The framework has four dimensions -- (T)rustworthiness, (A)uditability, (C)redibility and (T)ransferability. The development of TACT is…

Understanding the reason(s) for Differential Item Functioning (DIF) in the context of measurement is difficult. Although identifying potential DIF items is typically a statistical endeavor, understanding the reasons for DIF (and item repair or replacement) might require investigations that can be informed by qualitative work. Such work is…

The current review addresses the validity of published meta-analyses in education that determines the credibility and generalizability of study findings using a total of 56 meta-analyses published in education in the 2000s. Our objectives were to evaluate the current meta-analytic practices in education, identify methodological strengths and…

Due to the variations of design and protocol in qualitative inquiry, researchers may inadvertently create problems for themselves in terms of the trustworthiness of their research. Miscommunication between participants and researchers can especially arise from the unique and unpredictable nature of human dynamics. In this paper I contend that such…

The use of evaluation to examine and improve the quality of teaching and courses is now a component of most universities. However, despite the various methods and opportunities for evaluation, a lack of understanding of the processes, measures and value are some of the major impediments to effective evaluation. Evaluation requires an understanding…

The relevance of ethnographic research to evaluation of educational programs is discussed. The author focuses on evaluation questions that deal with finding out what is happening in a program and if what is happening is desirable. Ethnography suits the purposes of evaluators who answer such questions because they want to know how something works…

Evaluators need new strategies that will support their commitment to science and at the same time preserve the objectivity of the evaluation. The article discusses three strategies, dealing with: philosophical foundations, the subjectivity of the "objects" of the studies, and the subjectivity of the observer. (Author/GDC)

As with all media, the Internet structures and frames information, rewarding some information search and decision behaviors while punishing others and, thereby, strongly influences evaluation research results and possibilities. Now that the Internet is for many evaluators the information medium of choice, the impacts of the medium on evaluation…

Evaluation is viewed as essential to decision making and social policy development. Since conventional methods have been disappointing or inadequate, naturalistic inquiry (N/I) differs from conventional science in minimizing constraints on antecedent conditions (controls) and on output (dependent variables). N/I is phenomenological rather than…

The value of a multimethod, quantitative and qualitative approach to evaluation in an applied setting is demonstrated through a case study of an evaluation of a classroom literacy intervention (Classroom Publishing) that had been implemented without regard to subsequent evaluation. The two types of techniques complemented each other to create…

This paper presents criteria for establishing the trustworthiness of naturalistic inquiries, and specific techniques to facilitate their achievement or determine the degree of their achievement. The following criteria are briefly described: fairness; and ontological, educative, catalytic and tactical authenticity. Explored in greater detail,…

This paper examines how potential conflicts produced by ethical issues such as confidence and confidentiality can affect the research methodology of educational research and evaluation projects. Despite the differences in site selection, clientele, and relationship to treatment, both evaluators and researchers must make adjustments in method to…