As the development and application of large language models (LLMs) in physics education progress, the well-known AI-based chatbot ChatGPT4 has presented numerous opportunities for educational assessment. Investigating the potential of AI tools in practical educational assessment carries profound significance. This study explored the comparative…

The proliferation of large language models represents a paradigm shift in the landscape of automated essay scoring (AES) systems, fundamentally elevating their accuracy and efficacy. This study presents an extensive examination of large language models, with a particular emphasis on the transformative influence of transformer-based models, such as…

Grammatical knowledge for effective English writing remains a significant need among Thai university students, while identifying the relevant grammatical topics to address this need continues to be a challenge in teaching writing courses. Previous papers on writing in Thai universities have mainly focused on writing performance issues, rather than…

The aim of this project was to understand student perspectives on generative artificial intelligence (GAI) technologies such as Chat generative Pre-Trained Transformer (ChatGPT), in order to inform changes to the University of Liverpool Academic Integrity code of practice. The survey for this study was created by a library student team and vetted…

In university settings, writing argumentative essays from reading conflicting source texts is a common task for students. In performing this synthesis task, they must deal with conflicting claims about a controversial issue as they develop their own positions. Argumentative synthesis is characterized by writers' back-and-forth moves between…

Ethical considerations, including transparency, play an important role when using artificial intelligence (AI) in education. Explainable AI has been coined as a solution to provide more insight into the inner workings of AI algorithms. However, carefully designed user studies on how to design explanations for AI in education are still limited. The…

Academic writing is a key determinant to be successful in higher education institutions, and graduation requirements often involve students producing theses and dissertations using academic writing skills after conducting research. The COVID-19 pandemic has greatly affected pre-service teachers' learning in higher education institutions, and with…

This study examines the use of ChatGPT, in conjunction with other applications, Grammarly and Quillbot, as an automated writing evaluation (AWE) tool in a Recount and Narrative essay course for 18 English as a Foreign Language (EFL) students in Indonesia. The primary objective was to investigate the impact of these tools on the development of…

In recent years, there has been a growing trend of using informal styles in academic writing, including research articles. To examine the degree of formality in students' writing, this corpus-based study aimed to analyze the formal linguistic features in the academic writing assignments of English-major students at a Thai university. The learner…

This study aims to develop a robust rubric for evaluating artificial intelligence (AI)--assisted essay writing in English as a Foreign Language (EFL) contexts. Employing a modified Delphi technique, we conducted a comprehensive literature review and administered Likert scale questionnaires. This process yielded nine key evaluation criteria,…

This study explores the effectiveness of using ChatGPT, an Artificial Intelligence (AI) language model, as an Automated Essay Scoring (AES) tool for grading English as a Foreign Language (EFL) learners' essays. The corpus consists of 50 essays representing various types including analysis, compare and contrast, descriptive, narrative, and opinion…

This study seeks to clarify whether teachers are able to distinguish between essays written by English L2 students or generated by ChatGPT. 47 instructors who hold experience teaching English to native speakers of Japanese in universities or other higher education institutions were tested on whether they could identify between human written essays…

Students' writing proficiency is measured through holistic and analytical ratings in writing assessment; however, recent studies suggest that measurement of syntactic complexity in second language writing research has become an effective measure of writing proficiency. Within this paradigm, we investigated how automated measurement of syntactic…

Research has suggested that the type of feedback learners receive can impact on whether learners understand the feedback, the extent to which they engage with it, and whether they incorporate it in their revised drafts. However, to date, only a small number of studies have investigated learner engagement with corrective feedback provided by…

This report sets out the findings of the technical development work completed as part of the Use Cases for Generative AI in Education project, commissioned by the Department for Education (DfE) in September 2023. It has been published alongside the User Research Report, which sets out the findings from the ongoing user engagement activity…