Computing and Information Systems - Theses

Permanent URI for this collection

Search Results

Now showing 1 - 1 of 1
  • Item
    Thumbnail Image
    Anaphora Resolution in Procedural Text - from Domain to Domain
    Fang, Biaoyan ( 2022)
    Anaphora is an important and frequent concept in any form of discourse. It describes the use of expressions referring back to expressions used earlier in text, to avoid repetition. Anaphora resolution aims at resolving these reference relations in discourse and forms a core task in natural language understanding. It mainly contains two anaphoric types: coreference and bridging. While much effort has been targeted at anaphora resolution, most research has focused on these two anaphoric types separately. Specifically, anaphora research mostly focuses on coreference, modeling it from different perspectives across various resources. Bridging, on the other hand, has not been studied comprehensively. Different work analyzes bridging differently, leading to inconsistencies in bridging definitions. The lack of attention to bridging also brings challenges in capturing comprehensive anaphora phenomena in discourse -- only modeling coreference is not sufficient to capture complex anaphoric relations in text. It is becoming increasingly important to have both coreference and bridging annotated. Additionally, most existing anaphora research is based on declarative text. Procedural text, a common type of text, has received limited attention despite the richness and importance of anaphora phenomena in it, leaving much room for further exploration. In this thesis, we focus on anaphora resolution in procedural text, studying both coreference and bridging based on two common types of procedural text, chemical patents and recipes, and show that our proposed anaphora frameworks are well suited for procedural text. The four research questions we address in this thesis are: (1) How to model anaphora resolution in chemical patents? (2) How to combine different types of anaphora resolution? (3) How to incorporate external knowledge into anaphora resolution? (4) How to generalize our anaphora resolution model to domains apart from the biochemical domain? We address the first research question by proposing domain-specific anaphora annotation guidelines for chemical patents, targeting both coreference and bridging and incorporating general and domain-specific knowledge via in-depth investigations. We resolve ambiguities in bridging definitions by limiting the anaphoric relations to four specific subtypes related to the chemical domain while maintaining high coverage of anaphora phenomena. We achieve high IAA on the created ChEMU-Ref corpus, well above existing bridging corpora and demonstrating the reliability of the created dataset. To address the second research question, we propose an end-to-end joint training anaphora resolution model for coreference and bridging, adopting an end-to-end coreference resolution framework (Lee et al., 2017, 2018). Through empirical experiments on off-the-shelf anaphora corpora, we show the benefits of joint training for bridging. However, the impact on coreference is not clear. We argue that it could be due to ambiguity in the definition of bridging. To validate our hypothesis, we further experiment on two high-quality anaphora corpora with clear anaphora definitions, the ChEMU-Ref and RecipeRef (details in the last research question) datasets, and show the potential in improving both tasks through joint training, indicating the benefits of joint learning of coreference and bridging on high-quality anaphora corpora. Next, we address the third research question from the perspective of the utilization of pretrained language models based on the proposed end-to-end joint training framework, experimenting on the ChEMU-Ref corpus. We show that even with simple replacements, replacing generic language models (e.g. ELMo (Peters et al., 2018)) with domain pretrained language models (e.g. CHELMO (Zhai et al., 2019)), models achieve better performance, suggesting the potential of incorporating external knowledge for domain-specific anaphora resolution. Further explorations on recurrent neural network based and transformer based language models provide deeper insights, and suggest that different approaches might be needed to fully utilize different types of pretrained language models. For the last research question, we generalize the anaphora annotation framework developed for chemical patents to recipes with domain adjustments by detailed analysis of the similarities and differences between these two types of procedural text. Through in-depth comparison, we propose a more generic anaphora annotation framework for procedural text, designing in a hierarchy based on the state of entities. Based on the proposed annotation framework, we create the RecipeRef corpus, capturing rich anaphora phenomena in recipes, maintaining high IAA scores, and suggesting the feasibility of generalizing this framework to other procedural text. We observe further improvement from transfer learning, i.e. pretraining on the ChEMU-Ref dataset and fine-tuning on the RecipeRef dataset, indicating the transformation of general procedural knowledge in this domain. In summary, this thesis studies anaphora resolution in procedural text, particularly based on chemical patents and recipes, two common types of procedural text, and fills the gap in modeling and resolving anaphora resolution in this area. This establishes a firm base and contributes towards further research in anaphora resolution over procedural text.