Assessing students' answers and in particular natural language answers is a crucial challenge in the field of education. Advances in transformer-based models such as Large Language Models (LLMs), have led to significant progress in various natural language tasks. Nevertheless, amidst the growing trend of evaluating LLMs across diverse tasks,…

Though qualitative research has become more prevalent in practice over the last 30 years, there is still considerable uncertainty among researchers regarding how to ensure inter-rater consistency when teams are tasked with coding qualitative data. In this article, we offer an explanation of a methodology our qualitative team used to achieve…

For randomized trials that use text as an outcome, traditional approaches for assessing treatment impact require that each document first be manually coded for constructs of interest by trained human raters. This process, the current standard, is both time-consuming and limiting: even the largest human coding efforts are typically constrained to…

Linking data together can be a powerful way for governments and researchers alike to tackle vexing public policy research problems. However, for researchers, finding ways to link data directly between two departments can often be more challenging than even obtaining the data in the first place. Even when a researcher develops the necessary…

Reliable and valid assessment of parenting and child behaviors is critical for clinicians and researchers alike, and observational measures of parenting behaviors are often considered the gold standard for assessing parenting and parent-child interaction quality. The current study sought to evaluate the reliability and validity of the Coder…

Worked examples, which present an explained code for solving typical programming problems are among the most popular types of learning content in programming classes. Most approaches and tools for presenting these examples to students are based on line-by-line explanations of the example code. However, instructors rarely have time to provide…

Purpose: The aim of this study was to advance the analysis of written language transcripts by validating an automated scoring procedure using an automated open-access tool for calculating morphological complexity (MC) from written transcripts. Method: The MC of words in 146 written responses of students in fifth grade was assessed using two…

More than ever in the past, researchers have access to broad, educationally relevant text data from sources such as literature databases (e.g., ERIC), an open-ended response from online courses/surveys, online discussion forums, digital essays, and social media. These advances in data availability can dramatically increase the possibilities for…

The aim of this study was to explore how the Coding as Another Language (CAL) curriculum, developed by Boston College's DevTech Research Group and utilizing the ScratchJr app, impacted students' computational thinking, coding skills, and reading comprehension. To accomplish this, the research team randomly assigned thirteen schools in a…

Learning coding during early childhood is an effective way for children to practice computational thinking. Aspects of children's motivation can increase the likelihood that children approach computational thinking activities with enthusiasm and deep engagement. Gender inequities may interfere with children's readiness to take advantage of…

The aim of this study was to explore how the Coding as Another Language using ScratchJr (CAL-ScratchJr) curriculum, developed by Boston College's DevTech Research Group utilizing the ScratchJr app, impacted second grade students' computational thinking, coding skills, and reading comprehension. To accomplish this, the research team randomly…

Rural students often face challenges in receiving high-quality education in science, technology, engineering, and math (STEM). Yet without meaningful STEM educational opportunities, rural students might not develop the knowledge and skills needed to compete in a technology-driven workforce. The Learning by Making program (LbyM), an innovative…