ERIC Number: EJ1413716
Record Type: Journal
Publication Date: 2024
Pages: 21
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0218-8791
EISSN: EISSN-1742-6855
Available Date: N/A
The Comparison of General Tips for Mathematical Problem Solving Generated by Generative AI with Those Generated by Human Teachers
Asia Pacific Journal of Education, v44 n1 p8-28 2024
In designing an intelligent tutoring system, a core area of the application of AI in education, tips from the system or virtual tutors are crucial in helping students solve difficult questions in disciplines like mathematics. Traditionally, the manual design of general tips by teachers is time-consuming and error-prone. Generative AI, like ChatGPT, presents a new channel for designing general tips. This study utilized prompt engineering and Chain of Thought to summarize general tips for given mathematical problems (one geometry problem and one algebra problem) and their solutions. A Turing test was conducted to compare ChatGPT-generated general tips with human-designed ones. Results from 121 human evaluators, each assessing 6 ChatGPT-generated and 6 human-designed general tips for each of two mathematical problems, showed that the average score for ChatGPT-generated tips is less than that of human-designed tips at a statistically significant level (p < 0.05), and Zero-Shot CoT achieved the best score. However, no evaluator could distinguish the tip types exactly. The average precision, recall and F-value of all ChatGPT-generated tips are less than 40%. AI-generated general tips can serve as a valuable reference for teachers to enhance efficiency and students' mathematical learning.
Descriptors: Problem Solving, Artificial Intelligence, Learning Processes, Prompting, Instructional Design, Mathematics, Intelligent Tutoring Systems, Educational Technology, Influence of Technology, Technology Uses in Education, Mathematics Education, Testing
Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Author Affiliations: N/A