Voice interaction game design and gameplay

Abstract

This thesis is concerned with the phenomenon of voice-operated interaction with characters and environments in videogames. Voice interaction with virtual characters has become common in recent years, due to the proliferation of conversational user interfaces that respond to speech or text input through the persona of an intelligent personal assistant. Previous studies have shown that users experience a strong sense of social presence when speaking aloud to a virtual character, and that voice interaction can facilitate playful, social and imaginative experiences of the type that are often experienced when playing a videogame. Despite this, the user experience of voice interaction is frequently marred by frustration, embarrassment and unmet expectations.

The aim of this thesis is to understand how voice interaction can be used in videogames to support more enjoyable and meaningful player experiences. Voice-operated videogames have existed for more than three decades, yet little research exists on how they are designed and how they are received by players. The thesis addresses that knowledge gap through four empirical studies. The first study looks at player responses to a videogame character that can be given commands through a natural language interface. The second study is a historical analysis of voice-operated games that examines the technological and cultural factors that have shaped their form and popularity. The third study develops a pattern language for voice game design based on a survey of 471 published videogames with voice interaction features. The fourth study compares player responses to videogames that feature speech-based voice interaction and non-verbal voice interaction, and applies the theoretical perspective of frame analysis to interpret their reactions.

Through these studies, the thesis makes two main contributions to the human-computer interaction and games studies literature. First, it identifies five genres of voice gameplay that are based upon fundamentally different types of vocal activities, and details the design patterns and design goals that are distinctive to each genre. Second, it presents an empirically grounded theoretical model of gameplay that accounts for players’ feelings of engagement, social presence, frustration and embarrassment during voice gameplay. Overall, the thesis demonstrates that the fictional framing a videogame presents is a crucial factor in determining how players will experience its voice interaction features.