ERIC Number: EJ1186438
Record Type: Journal
Publication Date: 2018-Sep
Pages: 35
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1560-4292
EISSN: N/A
Available Date: N/A
A Multimodal Analysis of Making
Worsley, Marcelo; Blikstein, Paulo
International Journal of Artificial Intelligence in Education, v28 n3 p385-419 Sep 2018
This paper presents three multimodal learning analytic approaches from a hands-on learning activity. We use video, audio, gesture and bio-physiology data from a two-condition study (N = 20), to identify correlations between the multimodal data, experimental condition, and two learning outcomes: design quality and learning. The three approaches incorporate: 1) human-annotated coding of video data, 2) automated coding of gesture, audio and bio-physiological data and, 3) concatenated human-annotated and automatically annotated data. Within each analysis we employ the same machine learning and sequence mining techniques. Ultimately we find that each approach provides different affordances depending on the similarity metric and the dependent variable. For example, the analysis based on human-annotated data found strong correlations among multimodal behaviors, experimental condition, success and learning, when we relaxed constraints on temporal similarity. The second approach performed well when comparing students' multimodal behaviors as a time series, but was less effective using the temporally relaxed similarity metric. The take-away is that there are several strategies for doing multimodal learning analytics, and that many of these approaches can provide a meaningful glimpse into a complex data set, glimpses that may be difficult to identify using traditional approaches.
Descriptors: Multimedia Materials, Correlation, Outcomes of Education, Design, Video Technology, Nonverbal Communication, Data Analysis, Audio Equipment, Physiology, Experimental Groups, Learning Processes, Computer Assisted Instruction, Comparative Analysis, Student Behavior, Learning Activities
Springer. Available from: Springer Nature. 233 Spring Street, New York, NY 10013. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-348-4505; e-mail: customerservice@springernature.com; Web site: https://link-springer-com.bibliotheek.ehb.be/
Publication Type: Journal Articles; Reports - Evaluative
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Author Affiliations: N/A