ERIC Number: EJ1468099
Record Type: Journal
Publication Date: 2025-Jun
Pages: 22
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0735-6331
EISSN: EISSN-1541-4140
Available Date: 0000-00-00
Generative AI as a Coach to Help Students Enhance Proficiency in Question Formulation
Paul Kim1; Wilson Wang1; Curtis J. Bonk2
Journal of Educational Computing Research, v63 n3 p565-586 2025
Following the launch of the generative AI Web application, Ask.SMILE, designed to evaluate the cognitive levels of questions asked, 2559 educators generated 25,973 question-feedback sets over a three-month period, with an average of over 10 questions per participant. Analyses revealed a significant improvement in question quality from initial submissions to later stages. Specifically, participants progressed from Level 1 questions (simple recall) to Level 5 questions (creative and evaluative), based on Bloom's Taxonomy. However, participants who began with lower-quality questions showed varied progression paths to higher levels. To analyze the data, a Linear Mixed Model (LMM) was used to account for both fixed and random effects, revealing a statistically significant improvement in question quality over time (p < 0.01). ANOVA results further confirmed the significance of question stage on quality, and a Least Squares Means (LSMEANS) analysis indicated a precise and significant improvement in later-stage questions. Additionally, survey responses highlighted participants' appreciation for the ongoing development of AI-driven educational tools like Ask.SMILE. Several implications and future research directions are explored, including the potential to expand the platform's features beyond question evaluation.
Descriptors: Artificial Intelligence, Technology Uses in Education, Test Wiseness, Test Items, Test Construction, Technology Integration, Educational Technology, Taxonomy, Active Learning, Inquiry
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub-com.bibliotheek.ehb.be
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Author Affiliations: 1Stanford University, CA, USA; 2Indiana University, IN, USA