ERIC Number: EJ1466278
Record Type: Journal
Publication Date: 2025
Pages: 17
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: EISSN-2981-9954
Available Date: 0000-00-00
Evaluation of Automated Vocabulary Quiz Generation with VocQGen
Vocabulary Learning and Instruction, v14 n1 Article 2079 2025
VocQGen is an automated tool designed to generate multiple-choice cloze (MCC) questions for vocabulary assessment in second language learning contexts. It leverages several natural language processing (NLP) tools and OpenAI's GPT-4 model to produce MCC items quickly from user-specified word lists. To evaluate its effectiveness, we used the first sublist in the Academic Word List (AWL) to generate 60 questions with VocQGen. Then we compared the quality of 60 autogenerated questions with 40 manually created ones through expert reviews and through pilot testing with 68 students. Expert review results indicate that automatically generated questions exhibit higher grammatical accuracy and clearer contexts in question stems. However, the tool occasionally produces distractors that are acceptable as correct responses. Pilot testing results show that in general the number of correct responses is higher in autogenerated questions, indicating the less challenging nature of these questions. The study concludes that manual check is still required for questions generated by VocQGen and future work should focus on improving distractor effectiveness.
Descriptors: Vocabulary Skills, Artificial Intelligence, Computer Software, Multiple Choice Tests, Technology Uses in Education, Word Lists, Academic Language, Computer Assisted Testing, Language Tests, Natural Language Processing, Grammar, Accuracy, Cloze Procedure, Test Items, Comparative Analysis, Computational Linguistics, English (Second Language), Second Language Learning, Second Language Instruction, Teaching Methods, College Students, Foreign Countries
Castledown Publishers. Ground Level, 470 St Kilda Road, Melbourne, 3004, Australia. Tel: +61-3-7003-8355; e-mail: contact@castledown.com; Web site: https://www.castledown.com/journals/vli
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Japan
Grant or Contract Numbers: N/A
Author Affiliations: N/A