ERIC Number: EJ1441771
Record Type: Journal
Publication Date: 2024
Pages: 10
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: EISSN-2056-7936
Available Date: N/A
Evaluating Large Language Models in Analysing Classroom Dialogue
Yun Long; Haifeng Luo; Yu Zhang
npj Science of Learning, v9 Article 60 2024
This study explores the use of Large Language Models (LLMs), specifically GPT-4, in analysing classroom dialogue--a key task for teaching diagnosis and quality improvement. Traditional qualitative methods are both knowledge- and labour-intensive. This research investigates the potential of LLMs to streamline and enhance this process. Using datasets from middle school mathematics and Chinese classes, classroom dialogues were manually coded by experts and then analysed with a customised GPT-4 model. The study compares manual annotations with GPT-4 outputs to evaluate efficacy. Metrics include time efficiency, inter-coder agreement, and reliability between human coders and GPT-4. Results show significant time savings and high coding consistency between the model and human coders, with minor discrepancies. These findings highlight the strong potential of LLMs in teaching evaluation and facilitation.
Descriptors: Classroom Communication, Computational Linguistics, Chinese, Mathematics Instruction, Artificial Intelligence, Computer Software, Dialogs (Language), Time Management, Reliability, Teaching Methods, Middle School Students, Models, Comparative Analysis, Evaluation Methods, Instructional Effectiveness
Nature Portfolio. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://www.nature.com/npjscilearn/
Publication Type: Journal Articles; Reports - Research
Education Level: Junior High Schools; Middle Schools; Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Author Affiliations: N/A