ERIC Number: ED613663
Record Type: Non-Journal
Publication Date: 2021-Jun
Pages: 17
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
Measuring Conversational Uptake: A Case Study on Student-Teacher Interactions. EdWorkingPaper No. 21-424
Demszky, Dorottya; Liu, Jing; Mancenido, Zid; Cohen, Julie; Hill, Heather C.; Jurafsky, Dan; Hashimoto, Tatsunori
Annenberg Institute for School Reform at Brown University
In conversation, "uptake" happens when a speaker builds on the contribution of their interlocutor by, for example, acknowledging, repeating or reformulating what they have said. In education, teachers' uptake of student contributions has been linked to higher student achievement. Yet measuring and improving teachers' uptake at scale is challenging, as existing methods require expensive annotation by experts. We propose a framework for computationally measuring uptake, by (1) releasing a dataset of student-teacher exchanges extracted from US math classroom transcripts annotated for uptake by experts; (2) formalizing uptake as pointwise Jensen-Shannon Divergence (pJSD), estimated via next utterance classification; (3) conducting a linguistically-motivated comparison of different unsupervised measures and (4) correlating these measures with educational outcomes. We find that although repetition captures a significant part of uptake, pJSD outperforms repetition-based baselines, as it is capable of identifying a wider range of uptake phenomena like question answering and reformulation. We apply our uptake measure to three different educational datasets with outcome indicators. Unlike baseline measures, pJSD correlates significantly with instruction quality in all three, providing evidence for its generalizability and for its potential to serve as an automated professional development tool for teachers.
Descriptors: Interpersonal Communication, Teacher Student Relationship, Discourse Analysis, Measurement Techniques, Mathematics Education, Computation, Classification, Case Studies, Correlation, Models, Scores, Prediction, Dialogs (Language), Classroom Communication
Annenberg Institute for School Reform at Brown University. Brown University Box 1985, Providence, RI 02912. Tel: 401-863-7990; Fax: 401-863-1290; e-mail: AISR_Info@brown.edu; Web site: http://www.annenberginstitute.org
Publication Type: Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Annenberg Institute for School Reform at Brown University
Grant or Contract Numbers: N/A
Author Affiliations: N/A