Automated Evaluation and Application of Interactive Quality of Dialogue in Cross-Lingual and Non-Native Conversations
Loading...
Citations
Altmetric:
Altmetric::
Download
Associated Records
Authors
File Licence
Editors
Date
2026
Source Title
Journal Title
Journal ISSN
Volume Title
Publisher
Melbourne Authors
Affiliation
School of Languages and Linguistics
Document Type
PhD thesis
Subject Code
Citation
Access Status
Open Access
DOI
Access URL
Open Access URL
Open Access at PMC
Linked Resource URL
NHMRC Grant Code
ARC Grant Code
Description
© 2026 WEI GAO
Abstract
This thesis investigates how to automated evaluate non-native multi-lingual speakers engage in conversation communication and how their conversational competence can be automatically evaluated beyond surface-level grammatical accuracy via Large Language Models (LLM) in automated evaluation systems. While traditional language assessment frameworks emphasize linguistic correctness for second language (L2) in speaking assessment, this research argues that interactive competence (IC), such as the capacity to sustain, co-construct, and adapt dialogue, is an essential yet underexplored dimension of interactive and communicative competence in social contexts. And how this can be extended to dialogue system's interactive quality improvement via deployment of LLMs from single modality of multi-modality, single user to multi-speakers' engagement.
Drawing on interactional linguistics, we propose a multi-dimensional explainable evaluation framework for interactive dialogue conversations that integrates automation pipelines via LLM. The evaluation framework includes measures of turn-taking dynamics, topic management, and pragmatic social role appropriateness. Under proposed evaluation framework, this thesis investigate the cross-lingual transferability in conversation data, and with such explainable evaluation pipeline to improve the generation outcome of LLM in multilingual dialogue tasks.
A series of theoretical and empirical studies are then conducted in this thesis. Through discourse annotation and model analysis, the research identifies micro- and macro-linguistic features that differentiate proficient dialogic behavior, such as responsiveness, interactive alignment, and contextual flexibility. And build upon this evaluation pipeline, this study also investigate on improving LLMs from a human-like generation perspective via knowledge injection; and the proposed multilingual dialogue generation pipeline demonstrated improved interactive performance of LLM in multilingual dialogues. In addition, this thesis also examine employing proposed evaluation pipeline in multi-party moderated non-native conversational discussion and provided insights on L2 pedagogical design on how to improve L2 education in group communication settings.
The thesis offers a theoretically grounded and empirically validated framework for automated assessing interactive competence and helps to improve LLM enhanced interactive dialogue systems in multilingual contexts. It also provides pedagogical implications for designing communicative tasks and feedback systems that cultivate users’ interactive needs in dialogue agents.
Notes
Terms of Use
Keywords
Interactive Competence, Dialogue Evaluation, Multilingual NLP, Large Language Models, LLM Evaluation, Second Language Assessment, Language Assessment, Conversational AI, Cross-lingual Transfer, Pragmatics, Automated Evaluation