Students in a master's-level graduate course in school counseling program evaluation conducted a qualitative research study to investigate school counselors' experiences with evidence-based program delivery and evaluation. We discuss the study's findings that illuminated themes of challenges faced by school counselors with implications for…

Qualitative comparative analysis (QCA) is gaining ground in evaluation circles, but the number of applications is still limited. In this article, we consider the challenges that can emerge during a QCA evaluation by drawing on our experience of conducting one in the field of development cooperation. For each stage of the evaluation process, we…

The purpose of this paper is to introduce the Five-Level Qualitative Data Analysis (5LQDA) method for ATLAS.ti as a way to intentionally design methodological approaches applicable to the field of evaluation. To demonstrate my analytical process using ATLAS.ti, I use examples from an existing evaluation of a STEM Peer Learning Assistant program.…

Attention to mixed methods studies research has increased in recent years, particularly among funding agencies that increasingly require a mixed methods approach for program evaluation. At the same time, researchers operating within large-scale, rapid-turnaround research projects are faced with the reality that collection and analysis of large…

Realist evaluation (RE) is a research design increasingly used in program evaluation, that aims to explore and understand the influence of context and underlying mechanisms on intervention or program outcomes. Several methodological challenges, however, are associated with this approach. This article summarizes RE key principles and examines some…

Accountability and rigor in teacher education have been the focus of recent policy initiatives. Thus, data use practices have become increasingly critical to informing program improvement. Educational researchers have established self-study as a research methodology to intentionally be used by teacher educators to improve their practice. The…

Governments, organizations, and companies are generating copious amounts of data and analysis to support education decision-making around the world. While continued investments in data creation and management are necessary, the ultimate value of information is not in its "production," but its "use." Herein lies one of the…

Intended to guide scholars who are new to qualitative research, this methods brief focuses mostly on what reviewers look for in manuscripts submitted for publication. The author acknowledges that reviewers' preferences likely reflect their theoretical perspectives, research-oriented coursework, mentors, and research and writing experiences. The…

Purpose: The purposes of this paper are to provide a description of AI and to document and compare two applications of AI, one in program evaluation and another in an applied research study. Design/methodology/approach: Focus groups, interviews and observations were used to gather rich qualitative data which was used to detail Appreciative…

This article provides insight into how an adequate approach to selecting methods can establish credible and actionable evidence. The authors offer strategies to effectively support Extension professionals, including program developers and evaluators, in being more deliberate when selecting appropriate qualitative and quantitative methods. In…

Purpose: Novice researchers experience difficulties in analysing qualitative data. To develop the skills necessary for qualitative data analysis, theoretical manuals are often insufficient. Supervisors supporting students in analysing qualitative data stress the need for practical guidance, including exercises and feedback. The purpose of this…

With today's technology, Extension professionals have a variety of tools available for program evaluation. This article describes an innovative platform called VoiceThread that has been used in many classrooms but also is useful for conducting virtual focus group research. I explain how this tool can be used to collect qualitative participant…

In the five years following implementation of a chat reference service at James Madison University (JMU), the service proved very popular but was not closely assessed for quality of service. Using grounded theory and qualitative data analysis techniques, a comprehensive assessment effort was begun in earnest and is in progress. Preliminary results…

The Success By 6® (SB6) initiative is designed to support early care and education centers in improving and sustaining quality in Pennsylvania's Keystone STARS Quality Rating and Improvement System (QRIS). The SB6 evaluation report examines implementation and outcomes. The findings have implications for SB6 continous quality improvement process…

Christian school education is unique in nature. Integrating faith into academics led many families to choose an environment for their students that aligned with the faith, morals, and doctrinal beliefs they held as essential pieces in raising their children. The partnership with the school in instilling a strong foundation of faith for students…