Multimodal speech recognition errors and second language acquisition: An activity theoretic account

Abstract

Researchers in Computer-Aided Language Learning (CALL) have long recognized the potential benefits of Human Language Technology (HLT) for affording more naturalistic and authentic language learning contexts. Historically, CALL practitioners have argued that advancements in HLT would provide more intuitive, adaptive and compelling human-computer interfaces for effective language learning environments. However, CALL researchers have yet to demonstrate an understanding of how multimodal interaction affects the language learning process. Moreover, there are few Human-Computer Interaction (HCI) studies in CALL that critically examine the relationship between multimodal interaction, cognitive complexity and language acquisition.HCI methodologies in CALL have traditionally been couched within the cognitivist (information processing) paradigm. However, there is growing awareness amongst HCI practitioners of the importance of post-cognitivist, situated methodologies in modeling social systems, such as language learning. In contrast to cognitivist approaches, the goal of socially-oriented methodologies is to uncover emerging themes or patterns across subjective interpretations of user experiences. Such methodologies are therefore more relativistic and qualitative in nature. It is argued here that post-cognitivist HCI frameworks are more amenable to capturing the complexities of learner-computer interaction in CALL.This paper submits two contributions to the study of language learning and technology. Firstly, the paper presents initial findings from an exploratory study that undertook the development and evaluation of multimodal speech technology for second language vocabulary acquisition. Secondly, an activity theoretic examination of these findings is proffered. Findings from this study suggest future research agendas for the design and evaluation of multimodal systems in learner-centred environments.