ERIC Number: ED677279
Record Type: Non-Journal
Publication Date: 2025
Pages: 27
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: 2025-11-15
Natural Language Processing as a Scalable Method for Evaluating Educational Text Personalization by LLMs
Grantee Submission, Applied Sciences v15 Article 12128 2025
Four versions of science and history texts were tailored to diverse hypothetical reader profiles (high and low reading skills and domain knowledge), generated by four Large Language Models (i.e., Claude, Llama, ChatGPT, and Gemini). The Natural Language Processing (NLP) technique was applied to examine variations in Large Language Model (LLM) text personalization capabilities. NLP was leveraged to extract and quantify linguistic features of these texts, capturing linguistic variations as a function of LLMs, text genres, and reader profiles. An approach leveraging NLP-based analyses provides an automated and scalable solution for evaluating alignment between LLM-generated personalized texts and readers' needs. Findings indicate that NLP offers a valid and generalizable means of tracking linguistic variation in personalized educational texts, supporting its use as an evaluation framework for text personalization.
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Authoring Institution: N/A
IES Funded: Yes
Grant or Contract Numbers: R305T240035
Department of Education Funded: Yes
Author Affiliations: 1Learning Engineering Institute, Arizona State University, Tempe, AZ, USA

Peer reviewed
Direct link
