Background: Traditional survey efforts to gather outcome data at scale have significant limitations, including cost, time, and respondent burden. This pilot study explored new and innovative large-scale methods of collecting and validating data from publicly available sources. Taking advantage of emerging data science techniques, we leverage…

The purpose of this paper is to introduce the Five-Level Qualitative Data Analysis (5LQDA) method for ATLAS.ti as a way to intentionally design methodological approaches applicable to the field of evaluation. To demonstrate my analytical process using ATLAS.ti, I use examples from an existing evaluation of a STEM Peer Learning Assistant program.…

Members of the Data Science Program at George Washington University (GWU) designed and implemented a tuition-free two-week summer camp at GWU for high-school students from the Washington Metro Area. The United States Department of Agriculture (USDA) Office of the Chief Information Officer and his staff were our main partners in the project. The…

This project explored the reliability and utility of transcription in coding qualitative data across two studies in a program evaluation context. The first study tested the method of direct audio coding, or coding audio files without transcripts, using qualitative data software. The presence and frequency of codes applied in direct audio coding…

Objectives: In the USA, 18% of school-aged young people are classified as obese, and rural populations appear to be particularly at risk. Achieving high levels of fitness reduces the risk of obesity and underlying health conditions. To better understand youth obesity trends and fitness levels, annual fitness testing ([FT], that is, surveillance)…

We report on the implementation experience of carrying out data collection and other activities for a public health evaluation study on whether U.S. President's Emergency Plan for AIDS Relief (PEPFAR) investment improved utilization of health services and health system strengthening in Uganda. The retrospective study period focused on the PEPFAR…

This paper examines the development of data infrastructure in Australian schooling with a specific focus on interoperability standards that help to make new markets for education data. The conceptual framework combines insights from studies of infrastructure, economic markets and digital data. The case of the Australian National Schools…

Business schools continuously improve their processes and program assessment activities, but fall short in achieving faculty's awareness of process steps and disseminating results to inform and trigger continuous improvement actions. The assessment process of the Craig School of Business at California State University, Fresno had worked well. It…

This paper considers the use of log data provided by learning management systems when studying whether students obey the problem-based learning (PBL) method. Log analysis turns out to be a valuable tool in measuring the use of the learning material of interest. It gives reliable figures concerning not only the number of use sessions but also the…

A statewide survey of the infrastructure of teacher education program assessment systems in North Carolina, which include electronic portfolios as a component in the assessment system, measured their ability to meet current and anticipated future data demands for state approval and national accreditation. Almost two-thirds of the 46 teacher…

Maps can be a valuable addition to the Web analytics toolbox for Extension programs that use the Web to disseminate information. Extension professionals use Web analytics tools to evaluate program impacts. Maps add a unique perspective through visualization and analysis of geographic patterns and their relationships to other variables. Maps can…

Purpose: This paper aims to briefly outline the seamless evaluation approach and its application during an evaluation of ORIENT, a serious game aimed at young adults. Design/methodology/approach: In this paper, the authors detail a unobtrusive, embedded evaluation approach that occurs within the game context, adding value and entertainment to the…

Background: This article reports on an evaluation study of a project seeking to develop the use of position-linked datalogging with primary pupils in environmental science contexts. Purpose: The study sought to find out the extent to which the project had developed: (1) participant teachers' confidence in using datalogging as an everyday part of…

The debate surrounding teacher quality often fails to differentiate effectively between teacher-preparation providers. This failure also extends to distinguishing between teachers prepared in traditional campus-based accredited programs from those prepared in unaccredited campus-based programs. This paper compares assessment infrastructure and…

Learn more about a promising follow-up, participatory group process designed to document the results of Extension educational efforts within complex, real-life settings. The method, known as Ripple Effect Mapping, uses elements of Appreciative Inquiry, mind mapping, and qualitative data analysis to engage program participants and other community…