Computing and Information Systems - Theses
Permanent URI for this collection
Search Results
1 - 10 of 517
-
ItemLazy Constraint Generation and Tractable Approximations for Large Scale Planning Problems( 2023-12)In our research, we explore two orthogonal but related methodologies of solving planning instances: planning algorithms based on direct but lazy, incremental heuristic search over transition systems and planning as satisfiability. We address numerous challenges associated with solving large planning instances within practical time and memory constraints. This is particularly relevant when solving real-world problems, which often have numeric domains and resources and, therefore, have a large ground representation of the planning instance. Our first contribution is an approximate novelty search, which introduces two novel methods. The first approximates novelty via sampling and Bloom filters, and the other approximates the best-first search using an adaptive policy that decides whether to forgo the expansion of nodes in the open list. For our second work, we present an encoding of the partial order causal link (POCL) formulation of the temporal planning problems into a CP model that handles the instances with required concurrency, which cannot be solved using sequential planners. Our third significant contribution is on lifted sequential planning with lazy constraint generation, which scales very well on large instances with numeric domains and resources. Lastly, we propose a novel way of using novelty approximation as a polynomial reachability propagator, which we use to train the activity heuristics used by the CP solvers.
-
ItemRobust and Trustworthy Machine Learning( 2024-01)The field of machine learning (ML) has undergone rapid advancements in recent decades. The primary objective of ML models is to extract meaningful patterns from vast amounts of data. One of the most successful models, deep neural networks (DNNs), have been deployed in many real-world applications, such as face recognition, medical image analysis, gaming agents, autonomous driving and chatbots. Current DNNs, however, are vulnerable to adversarial perturbations, where an adversary can craft malicious perturbations to manipulate these models. For example, they can inject backdoor patterns into the training data, allowing them to control the model’s prediction with the backdoor pattern (known as a backdoor attack). Also, an adversary can introduce imperceptible adversarial noise to an image and change the prediction of a trained DNN with high confidence (known as an adversarial attack). These vulnerabilities of DNNs raise security concerns, particularly if deployed in safety-critical applications. The current success of DNNs relies on the volume of “free” data on the internet. A recent news article revealed that a company trains large-scale commercial models using personal data obtained from social media, which raises serious privacy concerns. This has led to an open question regarding whether or not data can be made unlearnable for DNNs. Unlike backdoor attacks, unlearnable data do not seek to control the model maliciously but only prevent the model from learning meaningful patterns in the data. Recent advancements in self-supervised learning (SSL) have shown promise in enabling models to learn from data without the need for human supervision. Annotating largescale datasets can be time-consuming and expensive, making SSL an attractive alternative. However, one challenge with SSL is the potential for dimensional collapse in the learned representations. This occurs when many features are highly correlated, giving rise to an “underfilling” phenomenon whereby the data spans only a lower-dimensional subspace. This can reduce the utility of a representation for downstream learning tasks. The first part of this thesis investigates defense strategies against backdoor attacks. Specifically, we develop a robust backdoor data detection method under the poisoning attacks threat model. We introduce a novel backdoor sample detection method Cognitive Distilation (CD). It extracts the minimal essence of features in the input image responsible for the model’s prediction. Through an optimization process, features that are not important are removed. For data containing backdoor triggers, only a small portion of semantic meaningless features are important for calssification, while clean data contains a larger amount of useful semantic features. Based on this characteristic, CD provides novel insights into existing attacks and can robustly detect backdoor samples. Additionally, the CD also reveals the connection between dataset bias and backdoor attacks. Through a case study, we show CD not only can detect bias matches with existing works but also discover several potential biases in a real-world dataset. The second part of this work examines the defences towards adversarial attacks. Adversarial training is one of the most effective defences. However, despite preliminary understandings developed for adversarial training, it is still not clear, from the architectural perspective, what configurations can lead to more robust DNNs. This work addresses this gap via a comprehensive investigation of the impact of network width and depth on the robustness of adversarially trained DNNs. The theoretical and empirical analysis provides the following insights: (1) more parameters do not necessarily help adversarial robustness; (2) reducing capacity at the last stage (the last group of blocks) of the network can improve adversarial robustness; and (3) under the same parameter budget, there exists an optimal architectural configuration for adversarial robustness. These architectural insights can help design adversarially robust DNNs. The third part of this thesis addresses the question of whether or not data can be made unexploitable for DNNs. This work introduces a novel concept, the unlearnable examples, which DNNs cannot learn useful features on such data. The unlearnable examples are generated through error-minimizing noise, which intentionally reduces the error of one or more of the training example(s) close to zero. Consequently, DNNs believe there is “nothing” worth learning from these example(s). The noise is restricted to be imperceptible to human eyes and thus does not affect normal data utility. This work demonstrates its flexibility under extensive experimental settings and practicability in a case study of face recognition. The fourth part of this thesis studies robust regularization techniques to address dimension collapse in SSL. Previous work has considered dimensional collapse at a global level. In this thesis, we demonstrate that learned representations can span over high dimensional space globally but collapse locally. To address this, we propose a method called local dimensional regularization (LDReg). Our formulation is based on the derivation of the Fisher-Rao metric to compare and optimize local distance distributions at an asymptotically small radius for each point. By increasing the local intrinsic dimensionality, we demonstrate through a range of experiments that LDReg improves the representation quality of SSL. The empirical results also show that LDReg can regularize dimensionality at both local and global levels. In summary, this work has contributed significantly toward robust and trustworthy machine learning. It includes the detection of backdoor samples, the development of robust architectures against adversarial examples, the introduction of unlearnable examples and a robust regularization to prevent dimension collapse in self-suerpvised learning.
-
ItemA Toolkit for Semantic Localisation Analysis( 2023-11)While UbiComp research has steadily improved the performance of localisation systems, the analysis of such datasets remains largely unaddressed. We present a tool to facilitate the querying and analysis of localisation time-series with a focus on semantic localisation. We developed a conceptual framework based on the idea of strongly-typed spaces, represented as symbolic coordinates. We also demonstrate its power and flexibility through an implementation of the framework and its application on a real-life case indoor localisation scenario.
-
ItemExplainable Computer Vision with Unsupervised Concept-based Explanations( 2023-10)This thesis focuses on concept-based explanations for deep learning models in the computer vision domain with unsupervised concepts. The success of deep learning methods significantly improves the performance of computer vision models. However, the quickly growing complexity of the models makes explainability a more important research focus. One of the major issues in computer vision explainability is that it is unclear what the appropriate features are that can be used in the explanations. Pixels are less understandable features compared with other domains like natural language processing with words as features. In recent years, concepts, that refer to the shared knowledge between human and AI systems with feature maps inside the deep learning model provide significant performance improvement as features in the explanations. Concept-based explanations become a good choice for explainability in computer vision. In most tasks, the supervised concept is the standard choice with better performance. Nevertheless, the concept learning task in supervised concept-based explanations additionally requires a dataset with a designed concept set and instance-level concept labels. Unsupervised concepts could reduce manual workload. In this thesis, we aim to reduce the performance gap between unsupervised and supervised concepts for concept-based explanations in computer vision. Targeting the baseline of concept bottleneck models (CBM) with supervised concepts, combined with the advances that unsupervised concepts do not require the concept set designing and labeling, the core contributions in this thesis make the unsupervised concepts an attractive alternative choice for concept-based explanations. Our core contributions are as follows: 1) We propose a new concept learning algorithm, invertible concept-based explanations (ICE). Explanations with unsupervised concepts can be evaluated with fidelity to the original model, like explanations with supervised concepts. Learned concepts are evaluated to be more understandable than baseline unsupervised concept learning methods like k-means clustering methods from ACE; 2) We propose a general framework of concept-based interpretable models with built-in faithful explanations similar to CBM. The framework makes the comparison between supervised and unsupervised concepts available. We show that unsupervised concepts provide competitive performance with model accuracy and concept interpretability; 3) We propose an example of applications using unsupervised concepts with counterfactual explanations, the fast concept-based counterfactual explanations (FCCE). In the ICE concept space, we propose the analytical solution to the counterfactual loss function. The calculation of counterfactual explanations in concept space takes less than 1e-5 seconds. Also, the FCCE is evaluated to be more interpretable through a human survey. In conclusion, previously, unsupervised concepts are not a choice for concept-based explanations as they suffer from many issues, such as being less interpretable and faithful to supervised concept-based explanations like CBM. With all our core contributions, the accuracy and interoperability performance of unsupervised concepts for concept-based explanations is improved to be competitive with supervised concept-based explanations. Since no extra requirements of concept set design and labeling are required, unsupervised concepts are an attractive choice for concept-based explanations in computer vision with competitive performance to supervised concepts. They also bring the benefit that no manual workload of concept set design and labeling is required.
-
ItemExplainable Computer Vision with Unsupervised Concept-based Explanations( 2023-10)This thesis focuses on concept-based explanations for deep learning models in the computer vision domain with unsupervised concepts. The success of deep learning methods significantly improves the performance of computer vision models. However, the quickly growing complexity of the models makes explainability a more important research focus. One of the major issues in computer vision explainability is that it is unclear what the appropriate features are that can be used in the explanations. Pixels are less understandable features compared with other domains like natural language processing with words as features. In recent years, concepts, that refer to the shared knowledge between human and AI systems with feature maps inside the deep learning model provide significant performance improvement as features in the explanations. Concept-based explanations become a good choice for explainability in computer vision. In most tasks, the supervised concept is the standard choice with better performance. Nevertheless, the concept learning task in supervised concept-based explanations additionally requires a dataset with a designed concept set and instance-level concept labels. Unsupervised concepts could reduce manual workload. In this thesis, we aim to reduce the performance gap between unsupervised and supervised concepts for concept-based explanations in computer vision. Targeting the baseline of concept bottleneck models (CBM) with supervised concepts, combined with the advances that unsupervised concepts do not require the concept set designing and labeling, the core contributions in this thesis make the unsupervised concepts an attractive alternative choice for concept-based explanations. Our core contributions are as follows: 1) We propose a new concept learning algorithm, invertible concept-based explanations (ICE). Explanations with unsupervised concepts can be evaluated with fidelity to the original model, like explanations with supervised concepts. Learned concepts are evaluated to be more understandable than baseline unsupervised concept learning methods like k-means clustering methods from ACE; 2) We propose a general framework of concept-based interpretable models with built-in faithful explanations similar to CBM. The framework makes the comparison between supervised and unsupervised concepts available. We show that unsupervised concepts provide competitive performance with model accuracy and concept interpretability; 3) We propose an example of applications using unsupervised concepts with counterfactual explanations, the fast concept-based counterfactual explanations (FCCE). In the ICE concept space, we propose the analytical solution to the counterfactual loss function. The calculation of counterfactual explanations in concept space takes less than 1e-5 seconds. Also, the FCCE is evaluated to be more interpretable through a human survey. In conclusion, previously, unsupervised concepts are not a choice for concept-based explanations as they suffer from many issues, such as being less interpretable and faithful to supervised concept-based explanations like CBM. With all our core contributions, the accuracy and interoperability performance of unsupervised concepts for concept-based explanations is improved to be competitive with supervised concept-based explanations. Since no extra requirements of concept set design and labeling are required, unsupervised concepts are an attractive choice for concept-based explanations in computer vision with competitive performance to supervised concepts. They also bring the benefit that no manual workload of concept set design and labeling is required.
-
ItemPractical declarative debugging of mercury programs(University of Melbourne, 2006)
-
ItemPractical declarative debugging of mercury programs(University of Melbourne, 2006)
-
ItemWord Associations as a Source of Commonsense Knowledge( 2023-12)Commonsense knowledge helps individuals naturally make sense of everyday situations and is important for AI systems to truly understand and interact with humans. However, acquiring such knowledge is difficult due to its implicit nature and sheer size, causing existing large-scale commonsense resources to suffer from a sparsity issue. This thesis addresses the challenge of acquiring commonsense knowledge by using word associations, a resource yet untapped for this purpose in natural language processing (NLP). Word associations are spontaneous connections between concepts that individuals make (e.g., smile and happy), reflecting the human mental lexicon. The aim of this thesis is to complement existing resources like commonsense knowledge graphs and pre-trained language models (PLMs), and enhance models’ ability to reason in a more intuitive and human-like manner. To achieve this aim, we explore three aspects of word associations: (1) understanding the relational knowledge they encode, (2) comparing the content and utility for NLP downstream tasks of large-scale word associations with widely-used commonsense knowledge resources, and (3) improving knowledge extraction from PLMs with word associations. We introduce a crowd-sourced large-scale dataset of word association explanations, which is crucial for disambiguating multiple reasons behind word associations. This resource fills a gap in the cognitive psychology community by providing a dataset to study the rationales and structures underlying associations. By automating the process of labelling word associations with relevant relations, we demonstrate that these explanations enhance the performance of relation extractors. We conduct a comprehensive comparison between large-scale word association networks and the ConceptNet commonsense knowledge graph, analysing their structures, knowledge content, and benefits for commonsense reasoning tasks. Even though we identify systematic differences between the two resources, we find that they both show improvements when incorporated into NLP models. Finally, we propose a diagnostic framework to understand the implicit knowledge encoded in PLMs and identify effective strategies for knowledge extraction. We show that word associations can enhance the quality of extracted knowledge from PLMs. The contributions of this thesis highlight the value of word associations in acquiring commonsense knowledge, offering insights into their utility in cognitive psychology and NLP research.
-
ItemA multistage computer model of picture scanning, image understanding, and environment analysis, guided by research into human and primate visual systems(University of Melbourne, Faculty of Engineering,, 1983)This paper describes the design and some testing of a computational model of picture scanning and image understanding (TRIPS), which outputs a description of the scene in a subset of English. This model can be extended to control the analysis of a three dimensional environment and changes of the viewing system's position within that environment. The model design is guided by a summary of neurophysiological, psychological, and psychophysical observations and theories concerning visual perception in humans and other primates, with an emphasis on eye movements. These results indicate that lower level visual information is processed in parallel in a spatial representation while higher level processing is mostly sequential, using a symbolic, post iconic, representation. The emphasis in this paper is on simulating the cognitive aspects of eye movement control and the higher level post iconic representation of images. The design incorporates several subsystems. The highest level control module is described in detail, since computer models Of eye movement which use cognitively guided saccade selection are not common. For other modules, the interfaces with the whole system and the internal computations required are out lined, as existing image processing techniques can be applied to perform these computations. Control is based on a production . system, which uses an "hypothesising" system - a simplified probabilistic associative production system - to determine which production to apply. A framework for an image analysis language (TRIAL), based on "THINGS". and "RELATIONS" is presented, with algorithms described in detail for the matching procedure and the transformations of size, orientation, position, and so On. TRIAL expressions in the productions are used to generate "cognitive expectations" concerning future eye movements and their effects which can influence the control of the system. Models of low level feature extraction, with parallel processing of iconic representations have been common in computer vision literature, as are techniques for image manipulation and syntactic and statistical analysis� Parallel and serial systems have also been extensively investigated. This model proposes an integration Of these approaches using each technique in the domain to which it is suited. The model proposed for the inferotemporal cortex could be also suitable as a model of the posterior parietal cortex. A restricted version of the picture scanning model (TRIPS) has been implemented, which demonstrates the consistency of the model and also exhibits some behavioural characteristics qualitatively similar to primate visual systems. A TRIAL language is shown to be a useful representation for the analysis and description of scenes. key words: simulation, eye movements, computer vision systems, inferotemporal, parietal, image representation, TRIPS, TRIAL.
-
ItemMulti-document Summarisation Supporting Clinical Evidence Review( 2023-12)Summarising (often contradictory) results of multiple clinical trials into conclusions which can be safely implemented by medical professionals in their daily practice is a very important, but highly challenging, task. In this thesis, we tackle it from three directions: we present our domain-specific evaluation framework, construct a new dataset for biomedical multi-document summarisation, and conduct experiments to analyse and improve the performance of summarisation models. We first examine what constitutes a well-formed answer to a clinical question, and define its three components -- PICO elements (biomedical entities), direction of findings, and modality (certainty). Next, we present a framework for human evaluation of biomedical summaries, which is based on these aspects and allows non-expert annotators to assess the factual correctness of conclusions faster and more robustly. Then, we use this framework to highlight issues with summarisation models, and examine the possibility of automating the summary evaluation using large generative language models. Following that, we present our multi-document summarisarion dataset which has several levels of inputs and targets granularity (such as documents, sentences, and claims) as well as rich annotation for the clinical evidence aspects we defined, and use it in several scenarios to test capabilities of existing models. Finally, we turn to the question of synthesing the input studies into conclusions, in particular, reflecting the direction and certainty of findings in summaries. First, we attempt to improve aggregation of entities and their relations using global attention mechanism in a pre-trained multi-document summarisation model. As this proves to be difficult, we examine if the models are at least able to detect modality and direction correctly. For that, we propose a dataset of counterfactual summaries and a method to test the models’ sensitivity to direction and certainty. Finally, we outline our preliminary experiments with a large generative language model, which shows some potential for better aggregation of direction values and PICO elements. Overall, the analysis and proposals in this thesis contribute deeper understanding of what is required of summarisation models to be able to generate useful and reliable multi-document summaries of clinical literature, improve their evaluation in that respect, and make a step towards better modeling choices.