ERIC Number: EJ1435693
Record Type: Journal
Publication Date: 2024
Pages: 20
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1055-3096
EISSN: EISSN-2574-3872
Available Date: N/A
Large Language Models as AI-Powered Educational Assistants: Comparing GPT-4 and Gemini for Writing Teaching Cases
Guido Lang; Tamilla Triantoro; Jason H. Sharp
Journal of Information Systems Education, v35 n3 p390-407 2024
This study explores the potential of large language models (LLMs), specifically GPT-4 and Gemini, in generating teaching cases for information systems courses. A unique prompt for writing three different types of teaching cases such as a descriptive case, a normative case, and a project-based case on the same IS topic (i.e., the introduction of blockchain technology in an insurance company) was developed and submitted to each LLM. The generated teaching cases from each LLM were assessed using subjective content evaluation measures such as relevance and accuracy, complexity and depth, structure and coherence, and creativity as well as objective readability measures such as Automated Readability Index, Coleman-Liau Index, Flesch-Kincaid Grade Level, Gunning Fog Index, Linsear Write Index, and SMOG Index. The findings suggest that while both LLMs perform well on objective measures, GPT-4 outperforms Gemini on subjective measures, indicating a superior ability to create content that is more relevant, complex, structured, coherent, and creative. This research provides initial empirical evidence and highlights the promise of LLMs in enhancing IS education while also acknowledging the need for careful proofreading and further research to optimize their use.
Descriptors: Computational Linguistics, Computer Software, Artificial Intelligence, Readability Formulas, Teaching Methods, Information Science Education, Information Systems, Case Method (Teaching Technique), Readability, Comparative Analysis, Proofreading, Measurement Techniques, Accuracy, Creativity
Journal of Information Systems Education. e-mail: editor@jise.org; Web site: http://www.jise.org
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: Flesch Kincaid Grade Level Formula
Grant or Contract Numbers: N/A
Author Affiliations: N/A