ERIC Number: EJ805776
Record Type: Journal
Publication Date: 2008-Sep
Pages: 23
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0267-1522
EISSN: N/A
Available Date: N/A
What Makes AS Marking Reliable? An Experiment with Some Stages from the Standardisation Process
Greatorex, Jackie; Bell, John F.
Research Papers in Education, v23 n3 p333-355 Sep 2008
It is particularly important that GCSE and A-level marking is valid and reliable as it affects the life chances of many young people in England. Current developments in marking technology are coinciding with potential changes in procedures to ensure valid and reliable marking. In this research the effectiveness of procedures to facilitate the reliability of marking were evaluated using a design with three interventions. Two of the interventions, the standardisation meeting and personal feedback, are similar to the conventional approach in GCSE and A-level practice but within the constraints of a research study. The other intervention, pre-written feedback, is how examples of marked scripts might be presented for part of examiner standardisation or training in other situations; it is also similar to the approach to feedback used in earlier research by Shaw and Baird et al. Each participating examiner experienced two of the three types of intervention. Marking data were gathered before and after the first intervention as well as after the second intervention. Inter-examiner reliability was analysed using ANOVA models. Due to the different levels of inter-examiner reliability at the beginning of the experiment it was hard to compare the effectiveness of the different interventions. Nevertheless, this research highlights that: (1) in line with previous research, standardisation meetings "on their own" are not particularly effective for "experienced" examiners; (2) as has been found in previous research, more experienced examiners were more lenient; (3) most combinations of interventions (combinations of standardisation processes--including the conventional approach) led to a decline in the size of the marking error; and (4) sometimes feedback (personal or pre-written) was effective and sometimes it was not. (Contains 3 figures, 9 tables and 3 notes.)
Descriptors: Scripts, Intervention, Interrater Reliability, Examiners, Foreign Countries, Grading, Standards, Feedback (Response), Error Patterns, Secondary Education
Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Reports - Research
Education Level: Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: United Kingdom (England)
Grant or Contract Numbers: N/A
Author Affiliations: N/A