Towards Multimodal, Interpretable and Robust Task-Oriented Dialogue Modeling
Loading...
Citations
Altmetric:
Altmetric::
Download
Associated Records
Authors
File Licence
Editors
Date
2022
Source Title
Journal Title
Journal ISSN
Volume Title
Publisher
Melbourne Authors
Affiliation
Computing and Information Systems
Document Type
PhD thesis
Subject Code
Citation
Access Status
Open Access
DOI
Access URL
Open Access URL
Open Access at PMC
Linked Resource URL
NHMRC Grant Code
ARC Grant Code
Description
© 2022 Shiquan Yang
Abstract
Over the last decade, task-oriented dialogue system has enabled canonical applications such as Amazon Alexa, Google Home, and Apple Siri to help humans accomplish specific tasks via natural language interfaces. Now while driving, we can simply ask Alexa to book a restaurant and check the weather forecast for us. Traditionally, these systems are built via the pipeline approach where different modules are separately trained and connected to generate responses. However, these systems suffer from the credit assignment issue, and requires labelled data for each component. Recently, end-to-end dialogue models have attracted increasing attention as they alleviate the drawbacks of the pipeline approach by directly mapping the user utterances to system responses. However, most existing end-to-end models leverage memory networks to generate responses, and they don't capture the structural information stored in the external knowledge base.
In this thesis, we first propose a novel end-to-end dialogue model that incorporates graph structure from the knowledge base. Observing that existing end-to-end models are limited as it uses a single modality (e.g., text only) knowledge base and multimodal data (e.g., images) can provide more information about the entities of interest (e.g., restaurants), we next introduce a multimodal dialogue system that can leverage a multimodal knowledge base. Nowadays, most end-to-end dialogue models use pre-trained language models as the backbone and although they are found to be empirically effective, their black-box nature means these dialogue systems do not provide explicit reasoning. To this end, we propose a novel explainable neuro-symbolic architecture for improving the interpretability of response generation. Lastly, we found that pre-trained model based dialogue models exploit shortcuts in the training data, learning the dataset instead of the task. To tackle this challenge, we propose debiasing techniques based on contrastive learning and adversarial filtering to improve model robustness. Extensive experimental studies demonstrate the effectiveness and robustness of our dialogue systems.
Notes
Terms of Use
Keywords
Task-Oriented Dialogue Systems, Graph Neural Networks, Multimodal, Interpretability, Robustness, Dialogue Systems, Neural Networks