Over the past decade, social scientists have adapted computational methods for qualitative text analysis, with the hope that they can match the accuracy and reliability of hand coding. The emergence of GPT and open-source generative large language models (LLMs) has transformed this process by shifting from programming to engaging with models using…

Generative AI (GenAI) is quickly becoming a valuable tool for sociological research. Already, sociologists employ GenAI for tasks like classifying text and simulating human agents. We point to another major use case: the extraction of structured information from unstructured text. Information Extraction (IE) is an established branch of Natural…

Educators and researchers are interested in ways that ChatGPT and other generative AI tools might move beyond the role of "cheatbot" and become part of the network of resources students use for writing. We studied how high school students used ChatGPT as a writing support while writing arguments about topics like school mascots. We…

Open-text responses provide researchers and educators with rich, nuanced insights that multiple-choice questions cannot capture. When reliably assessed, such responses have the potential to enhance teaching and learning. However, scaling and consistently capturing these nuances remain significant challenges, limiting the widespread use of…

Linguistic phenomena (e.g., words and syntactic structure) co-occur with a wide variety of meanings. These systematic correlations can help readers to interpret a text and create predictions about upcoming material. However, to what extent these correlations influence discourse processing is still unknown. We address this question by examining…

As artificial intelligence and natural language processing methods rapidly develop, communication plays a pivotal role in every-day interactions. In this theoretical paper, we explore the overlap and commonalities between question-asking and prompt engineering. While seemingly distinct, these processes share a common foundation in essential skills…

Using a quasi-experimental method and content analysis as a technique, this study tests ChatGPT, in its version 4, by assessing its textual characteristics and overall understanding regarding the recognition criteria of provisions under International Accounting Standards (IAS) 37, as issued by the International Accounting Standards Board (IASB).…

Large Language Models (LLMs) present a radically new paradigm for the study of "information foraging behavior." We study how LLM technology is used for pedagogical content creation by a sample of 25 participants in a doctoral-level Artificial Intelligence (AI) in Education course, and the role of computational-thinking skills in shaping…

It's clear that the transformative potential of artificial intelligence (AI) programs such as ChatGPT (generative pre-trained transformer) will significantly impact education, and it is natural for teachers to feel apprehensive about this change. This article explores the role of ChatGPT in a high school English classroom and discusses how it can…

We conducted two experiments to assess the alignment between Generative AI (GenAI) text personalization and hypothetical readers' profiles. In Experiment 1, four LLMs (i.e., Claude 3.5 Sonnet; Llama; Gemini Pro 1.5; ChatGPT 4) were prompted to tailor 10 science texts (i.e., biology, chemistry, physics) to accommodate four different profiles…

An increasing number of studies have investigated how ChatGPT can aid in written assessment and feedback provision. However, many studies overlook its conversational design and underlying architecture, raising concerns about the reliability and validity of their analytical outputs. Therefore, applying first principles thinking to prompt use, and…

A longstanding criticism of traditional high-stakes writing assessments is their use of static prompts in which test takers compose a single text in response to a prompt. These static prompts do not allow measurement of the writing process. This paper describes the development and validation of an innovative interactive writing task. After the…

This paper systematically explores how Large Language Models (LLMs) generate explanations of code examples of the type used in intro-to-programming courses. As we show, the nature of code explanations generated by LLMs varies considerably based on the wording of the prompt, the target code examples being explained, the programming language, the…

This paper describes the results of an archival desk-study that analyzed worksheets produced by four students using ChatGPT as a coach. ChatGPT is a Generative Pre-Trained Large Language model that can write comprehensively in various languages and styles. It was discovered that it could pass university level physics exams and perform at the level…

The value of generative AI tools in higher education has received considerable attention. Although there are many proponents of its value as a learning tool, many are concerned with the issues regarding academic integrity and its use by students to compose written assessments. This study evaluates and compares the output of three commonly used…