Two of the central issues in comparative judgement (CJ), which are perhaps underexplored compared to questions of the method's reliability and technical quality, are "what processes do judges use to make their decisions" and "what features do they focus on when making their decisions?" This article discusses both, in the…

Conversational Intelligent Tutoring Systems (ITSs) are expensive to develop. While simple online courseware could be easily authored by teachers, the authoring of conversational ITSs usually involves a team of experts with different expertise, including domain experts, linguists, instruction designers, programmers, artists, computer scientists,…

AutoTutor uses conversational intelligent agents in learning environments. One of the major challenges in developing AutoTutor applications is to assess students' natural language answers to AutoTutor questions. We investigated an AutoTutor dataset with 3358 student answers to 49 AutoTutor questions. In comparisons with human ratings, we found…

In the UK and elsewhere, large-scale educational assessment agencies are shifting the mode of school examination marking towards having examiners mark examination scripts on screen rather than on paper. This shift has prompted questions about whether the mode of marking might influence examiner marking accuracy, particularly in relation to…

UK awarding bodies increasingly use technology to distribute digital copies of students' examination scripts to examiners for marking. This process allows senior examiners (team leaders) to remotely monitor and feedback on the marking quality of examiners under their supervision throughout the marking period. The objective of this study was to…

It is particularly important that GCSE and A-level marking is valid and reliable as it affects the life chances of many young people in England. Current developments in marking technology are coinciding with potential changes in procedures to ensure valid and reliable marking. In this research the effectiveness of procedures to facilitate the…

Tutor attitudes toward the marking of a sample of scripts from a particular course at the Open University were discussed at meetings called the Tutor Agreement Trial. The main object was to lessen the feeling of isolation often experienced by part-time staff teaching at a distance. (LBH)

Instruction can be systematically designed using the procedures, or model, set forth in this handbook. Steps are given for writing, selecting, and organizing behavioral objectives and for identifying their levels. Test construction, testing, and grading are explained. Flowcharts illustrate how to determine required competencies and actual entering…