ERIC Number: ED656777
Record Type: Non-Journal
Publication Date: 2021-Sep-27
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
Can Virtual Replace In-Person Coaching? Experimental Evidence on Teacher Professional Development and Student Learning
Jacobus Cilliers; Brahm Fleisch; Janeli Kotze; Nompumelelo Mohohlwane; Stephen Taylor
Society for Research on Educational Effectiveness
Background: Information Communication Technology (ICT) holds the promise of enabling low-cost teacher professional development at scale. An expert coach, for example, could reach far more teachers virtually, thus reducing salary and transport costs. But the benefits of in-person interaction--such as developing relationships of trust and accountability--might be difficult to replicate. Research Question: We ask whether virtual coaching can be equally effective at improving teaching practices and student learning, compared to in-person coaching. As also ask which modality is most cost-effective. Setting and population: The setting is 230 public primary schools in poor communities in Limpopo province, South Africa. Students in these schools either speak isiZulu or Siswati as a home language, and English is only taught as a second language. Intervention: We evaluate two different modalities of providing continuous teacher professional development for teaching English as a Second Language (ESL)--on-site vs virtual coaching. In both programs teachers received the same learning materials and training at the start of the year, and the curriculum and content of the lesson plans were the same and fully aligned with government curriculum. However, the on-site coaching intervention differed from the virtual coaching in two important dimensions. First, teachers in the on-site program would receive in-classroom visits by a coach, whereas teachers in the other program would interact virtually with a coach through phone calls, regular text messages, WhatsApp groups, and participation in competitions. Second, the format of the daily lesson plans was paper-based in the on-site coaching intervention but was on an electronic tablet in the virtual coaching intervention. These programs were implemented over a period of three years, targeting the teachers assigned to a different grade each year (grade one teachers in the first year, grade two teachers in the second year, etc). Research Design: We randomly assigned 50 schools to each of the treatment groups and 80 schools to the control. In addition, we randomly sampled and assessed 20 grade one students per school before the start of the program in February 2017. We then tracked the same cohort of students over a period of three years, starting in February 2017 when they entered grade one, and ending in November 2019. At the end of every school year these students were assessed and their teachers surveyed. We also performed classroom observations in a sub-set of 53 schools at the end of the third year. The sample is balanced at baseline, and student attrition (18% after three years) is balanced across treatment arms. Data collection and analysis: We combined different subtasks of student learning into two indices--English oral language proficiency and English reading proficiency--using principal components. Oral language focuses on vocabulary development and consists of the English expressive vocabulary and the English listening comprehension subtask. English reading proficiency consists of the following subtasks: English word recognition, English oral reading fluency, English reading comprehension and English written comprehension. These are two separate skills that students should be able to master by the end of grade 3. We also constructed an index for home language reading proficiency, to test for spillover effects. Findings/Results: We find that, after three years, on-site coaching improved students' English oral language and reading proficiency (0.31 and 0.13 SD, respectively). Virtual coaching had a smaller impact on English oral language proficiency (0.12 SD), no impact on English reading proficiency, and an unintended negative effect on home language literacy. Classroom observations show that on-site coaching improved teaching practices, and virtual coaching led to larger crowding-out of home language teaching time. On-site coaching is more cost-effective. Moreover, tablet usage data show that technology itself was not a barrier to program implementation, since almost all the teachers in the virtual arm used tablets and accessed the lesson plans. Rather, the pattern of tablet usage--which was better earlier in the term, and highest in the week in which teachers were expected to submit assessment results--suggest that the binding constraint was teacher motivation or their ability to keep pace with the curriculum, rather than the technology itself. Consistent with this interpretation, teachers in the on-site coaching intervention were far more likely than teachers in the virtual coaching intervention to mention the coach as someone who holds them accountable and provides pedagogical support. The same service provider implemented both programs, and the quality of implementation was equally high for all three years of the study. Conclusions: The main finding of this paper is sobering--a virtual coaching alternative, which was somewhat less expensive and considerably less reliant on human resources, did not have the same desired effect, and actually reduced home language literacy. The research agenda to design innovative programs that allow meaningful support to teachers at a large scale must continue. We draw two conclusions: (1) Strong complementarities exist between technological interventions and the incentives faced by those who are required to adopt the technology. Technology provides opportunities to improve teacher productivity, provided that the teachers face the appropriate incentives to apply these technologies. Although the virtual coach can provide the same technical input as an on-site coach, they cannot provide the same level of accountability, since they are not directly monitoring the teachers in the classroom. Effective virtual teacher continuous professional development programs need to put structures in place to encourage and motivate teachers to participate. (2) Most likely, a more effective coaching program should involve a combination of some initial face-to-face coaching to establish the relationship, followed up with virtual coaching to sustain the instructional practice change. Moreover, teachers need to share video recordings of their teaching to the coach, in order to receive targeted feedback.
Descriptors: Faculty Development, Electronic Learning, Coaching (Performance), Distance Education, Poverty, Language Teachers, English (Second Language), Second Language Instruction, Foreign Countries, Program Effectiveness, Elementary School Teachers, In Person Learning, Educational Technology, Elementary School Students, Language Skills, Reading Skills
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Reports - Research
Education Level: Elementary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Identifiers - Location: South Africa
Grant or Contract Numbers: N/A
Author Affiliations: N/A

Peer reviewed
Direct link
