ERIC Number: EJ1251422
Record Type: Journal
Publication Date: 2020-Jun
Pages: 22
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-2211-1662
EISSN: N/A
Available Date: N/A
Predicting Group Performance Using Process Data in a Collaborative Assessment
Mohan, Kaushik; Bergner, Yoav; Halpin, Peter
Technology, Knowledge and Learning, v25 n2 p367-388 Jun 2020
Technology-based assessments that involve collaboration among students offer many sources of process data, although it remains unclear which aspects of these data are most meaningful for making inferences about students' collaborative skills. Recent research has focused mainly on theory-based rubrics for qualitative coding of process data (e.g., text from chat dialogues, click-stream data), but many reliability and validity issues arise in the application of such rubrics. In this research, we take a more data-driven approach to the problem. Data were collected from 122 dyads who interacted over online chat to complete a twelfth-grade mathematics assessment. We focus on features of chat and click-stream that can be extracted automatically, including the extent to which chat dialogue contained content from assessment materials; chat-based cues of affective tone and mirroring; and temporal synchronization in task-related activities. Using a block-wise linear regression, we show that process features of chat and click-stream accounted for 30.5% of the variation in group performance, after controlling for group members math proficiency and the total number of words in the chat dialogue. The full model explained 61% of the variation in group performance. Implications for the design and scoring of collaborative assessments are discussed.
Descriptors: Computer Assisted Testing, Student Evaluation, Cooperation, Grade 12, Mathematics Tests, Synchronous Communication, Performance, Data Use, Prediction
Springer. Available from: Springer Nature. 233 Spring Street, New York, NY 10013. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-348-4505; e-mail: customerservice@springernature.com; Web site: https://link-springer-com.bibliotheek.ehb.be/
Publication Type: Journal Articles; Reports - Research
Education Level: Grade 12; High Schools; Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Author Affiliations: N/A