ERIC Number: ED672055
Record Type: Non-Journal
Publication Date: 2022-Jun
Pages: 34
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: 0000-00-00
Assessors Influence Results: Evidence on Enumerator Effects and Educational Impact Evaluations. EdWorkingPaper No. 22-586
Daniel Rodriguez-Segura; Beth E. Schueler
Annenberg Institute for School Reform at Brown University
A significant share of education and development research uses data collected by workers called "enumerators." It is well-documented that "enumerator effects"--or inconsistent practices between the individual people who administer measurement tools-- can be a key source of error in survey data collection. However, it is less understood whether this is a problem for academic assessments or performance tasks. We leverage a remote phone-based mathematics assessment of primary school students and survey of their parents in Kenya. Enumerators were randomized to students to study the presence of enumerator effects. We find that both the academic assessment and survey was prone to enumerator effects and use simulation to show that these effects were large enough to lead to spurious results at a troubling rate in the context of impact evaluation. We therefore recommend assessment administrators randomize enumerators at the student level and focus on training enumerators to minimize bias. [This work has been supported by Innovations for Poverty Action (IPA).]
Descriptors: Data Collection, Educational Research, Research Problems, Elementary School Students, Parents, Telecommunications, Mathematics Tests, Outcome Measures, Foreign Countries, Bias
Annenberg Institute for School Reform at Brown University. Brown University Box 1985, Providence, RI 02912. Tel: 401-863-7990; Fax: 401-863-1290; e-mail: annenberg@brown.edu; Web site: https://annenberg.brown.edu/
Publication Type: Reports - Research; Tests/Questionnaires
Education Level: Elementary Education
Audience: N/A
Language: English
Authoring Institution: Annenberg Institute for School Reform at Brown University
Identifiers - Location: Kenya
Grant or Contract Numbers: NWU0004X1
Author Affiliations: N/A