Generative artificial intelligence technologies, such as ChatGPT, bring an unprecedented change in education by leveraging the power of natural language processing and machine learning. Employing ChatGPT to assist with marking written assessment presents multiple advantages including scalability, improved consistency, eliminating biases associated…

Quantitative measures of systematic differences in OSCE scoring across examiners (often termed examiner stringency) can threaten the validity of examination outcomes. Such effects are usually conceptualised and operationalised based solely on checklist/domain scores in a station, and global grades are not often used in this type of analysis. In…

The technology of automated short answer grading (ASAG) can efficiently process answers according to human-prepared grading examples. Computer-assisted acquisition of grading examples uses a computer algorithm to sample real student responses for potentially good examples. The process is critical for optimizing the grading accuracy of machine…

This paper presents the case of an adjunct university professor to illustrate the dilemma of using artificial intelligence (AI) technology to grade student papers. The hypothetical case discusses the benefits of using a commercial AI service to grade student papers--including discretion, convenience, pedagogical merits of consistent feedback for…

The purpose of this qualitative phenomenological study was to explore Pennsylvania public high school principals' leadership of grading practices reform and how that leadership was influenced by enabling and constraining factors within schools' cultures and communities. The analysis of the accounts of the lived experiences of 10 principals…

This study uses the concept of dilemmatic space to unpack the complexities of teachers' work when it comes to assessing students in the GenAI age. A key idea of dilemmatic space is that dilemmas are not 'out there' but constructions based on individuals' priorities, knowledge and values. Therefore, studying what teachers perceive as 'dilemmatic'…

This study explores the impact of three alternative grading systems--specifications grading, standards-based grading (SBG), and ungrading--on student motivation, engagement, stress, enjoyment, and perceptions of fairness. Quantitative and qualitative analyses were conducted with students from mathematics, computer science, and statistics courses.…

High-quality feedback requires understanding of a student's work, insights into what concepts would help them improve, and language that matches the preferences of the specific teaching team. While Large Language Models (LLMs) can generate coherent feedback, adapting these responses to align with specific teacher preferences remains an open…

We conducted a systematic literature review on automated grading and feedback tools for programming education. We analysed 121 research papers from 2017 to 2021 inclusive and categorised them based on skills assessed, approach, language paradigm, degree of automation, and evaluation techniques. Most papers assess the correctness of assignments in…

This study evaluates the ability of large language models (LLMs) to deliver criterion-based grading and examines the impact of prompt engineering with detailed criteria on grading. Using well-established human benchmarks and quantitative analyses, we found that even free LLMs achieve criterion-based grading with a detailed understanding of the…

Research on faculty emotions is scarce, despite their evident relevance for faculty well-being, higher education quality, and student outcomes. The present studies aimed to investigate six discrete emotions (enjoyment, pride, boredom, anxiety, anger, frustration) faculty may experience during grading. Study 1 compared faculty emotions for grading…

"Stochastic programs," which are programs that produce probabilistic output, are a pivotal paradigm in various areas of CS education from introductory programming to machine learning and data science. Despite their importance, the problem of automatically grading such programs remains surprisingly unexplored. In this paper, we formalize…

This paper proposes to evaluate learning efficiency by implementing the flipped classroom and automatic source code evaluation based on the Kirkpatrick evaluation model in students of CS1 programming course. The experimentation was conducted with 82 students from two CS1 courses; an experimental group (EG = 56) and a control group (CG = 26). Each…

This work tested the hypothesis that the academic performance of students in a distance learning course in Biological Sciences (PBS-CEDERJ) decreased during the COVID-19 pandemic in disciplines with and without practical activities. Performance was quantified via students' grades in PBS-CEDERJ disciplines in pre-pandemic years (2018 and 2019) and…

We propose a novel method for automated short answer grading (ASAG) designed for practical use in real-world settings. The method combines LLM embedding similarity with a nonlinear regression function, enabling accurate prediction from a small number of expert-graded responses. In this use case, a grader manually assesses a few responses, while…