Over the past decade, social scientists have adapted computational methods for qualitative text analysis, with the hope that they can match the accuracy and reliability of hand coding. The emergence of GPT and open-source generative large language models (LLMs) has transformed this process by shifting from programming to engaging with models using…

Generative AI (GenAI) is quickly becoming a valuable tool for sociological research. Already, sociologists employ GenAI for tasks like classifying text and simulating human agents. We point to another major use case: the extraction of structured information from unstructured text. Information Extraction (IE) is an established branch of Natural…

Educators and researchers are interested in ways that ChatGPT and other generative AI tools might move beyond the role of "cheatbot" and become part of the network of resources students use for writing. We studied how high school students used ChatGPT as a writing support while writing arguments about topics like school mascots. We…

Open-text responses provide researchers and educators with rich, nuanced insights that multiple-choice questions cannot capture. When reliably assessed, such responses have the potential to enhance teaching and learning. However, scaling and consistently capturing these nuances remain significant challenges, limiting the widespread use of…

As artificial intelligence and natural language processing methods rapidly develop, communication plays a pivotal role in every-day interactions. In this theoretical paper, we explore the overlap and commonalities between question-asking and prompt engineering. While seemingly distinct, these processes share a common foundation in essential skills…

Large Language Models (LLMs) present a radically new paradigm for the study of "information foraging behavior." We study how LLM technology is used for pedagogical content creation by a sample of 25 participants in a doctoral-level Artificial Intelligence (AI) in Education course, and the role of computational-thinking skills in shaping…

We conducted two experiments to assess the alignment between Generative AI (GenAI) text personalization and hypothetical readers' profiles. In Experiment 1, four LLMs (i.e., Claude 3.5 Sonnet; Llama; Gemini Pro 1.5; ChatGPT 4) were prompted to tailor 10 science texts (i.e., biology, chemistry, physics) to accommodate four different profiles…

An increasing number of studies have investigated how ChatGPT can aid in written assessment and feedback provision. However, many studies overlook its conversational design and underlying architecture, raising concerns about the reliability and validity of their analytical outputs. Therefore, applying first principles thinking to prompt use, and…

A longstanding criticism of traditional high-stakes writing assessments is their use of static prompts in which test takers compose a single text in response to a prompt. These static prompts do not allow measurement of the writing process. This paper describes the development and validation of an innovative interactive writing task. After the…

Heightened concerns about the health of the teaching profession highlight the importance of studying the early teacher pipeline. This exploratory, descriptive article examines preservice teachers' expressed motivation for pursuing a teaching career. Using data from a large teacher education program in Texas, we use a natural language processing…