Computing and Information Systems - Theses
Permanent URI for this collection
Search Results
1 - 3 of 3
-
ItemExplainable Computer Vision with Unsupervised Concept-based Explanations( 2023-10)This thesis focuses on concept-based explanations for deep learning models in the computer vision domain with unsupervised concepts. The success of deep learning methods significantly improves the performance of computer vision models. However, the quickly growing complexity of the models makes explainability a more important research focus. One of the major issues in computer vision explainability is that it is unclear what the appropriate features are that can be used in the explanations. Pixels are less understandable features compared with other domains like natural language processing with words as features. In recent years, concepts, that refer to the shared knowledge between human and AI systems with feature maps inside the deep learning model provide significant performance improvement as features in the explanations. Concept-based explanations become a good choice for explainability in computer vision. In most tasks, the supervised concept is the standard choice with better performance. Nevertheless, the concept learning task in supervised concept-based explanations additionally requires a dataset with a designed concept set and instance-level concept labels. Unsupervised concepts could reduce manual workload. In this thesis, we aim to reduce the performance gap between unsupervised and supervised concepts for concept-based explanations in computer vision. Targeting the baseline of concept bottleneck models (CBM) with supervised concepts, combined with the advances that unsupervised concepts do not require the concept set designing and labeling, the core contributions in this thesis make the unsupervised concepts an attractive alternative choice for concept-based explanations. Our core contributions are as follows: 1) We propose a new concept learning algorithm, invertible concept-based explanations (ICE). Explanations with unsupervised concepts can be evaluated with fidelity to the original model, like explanations with supervised concepts. Learned concepts are evaluated to be more understandable than baseline unsupervised concept learning methods like k-means clustering methods from ACE; 2) We propose a general framework of concept-based interpretable models with built-in faithful explanations similar to CBM. The framework makes the comparison between supervised and unsupervised concepts available. We show that unsupervised concepts provide competitive performance with model accuracy and concept interpretability; 3) We propose an example of applications using unsupervised concepts with counterfactual explanations, the fast concept-based counterfactual explanations (FCCE). In the ICE concept space, we propose the analytical solution to the counterfactual loss function. The calculation of counterfactual explanations in concept space takes less than 1e-5 seconds. Also, the FCCE is evaluated to be more interpretable through a human survey. In conclusion, previously, unsupervised concepts are not a choice for concept-based explanations as they suffer from many issues, such as being less interpretable and faithful to supervised concept-based explanations like CBM. With all our core contributions, the accuracy and interoperability performance of unsupervised concepts for concept-based explanations is improved to be competitive with supervised concept-based explanations. Since no extra requirements of concept set design and labeling are required, unsupervised concepts are an attractive choice for concept-based explanations in computer vision with competitive performance to supervised concepts. They also bring the benefit that no manual workload of concept set design and labeling is required.
-
ItemExplainable Computer Vision with Unsupervised Concept-based Explanations( 2023-10)This thesis focuses on concept-based explanations for deep learning models in the computer vision domain with unsupervised concepts. The success of deep learning methods significantly improves the performance of computer vision models. However, the quickly growing complexity of the models makes explainability a more important research focus. One of the major issues in computer vision explainability is that it is unclear what the appropriate features are that can be used in the explanations. Pixels are less understandable features compared with other domains like natural language processing with words as features. In recent years, concepts, that refer to the shared knowledge between human and AI systems with feature maps inside the deep learning model provide significant performance improvement as features in the explanations. Concept-based explanations become a good choice for explainability in computer vision. In most tasks, the supervised concept is the standard choice with better performance. Nevertheless, the concept learning task in supervised concept-based explanations additionally requires a dataset with a designed concept set and instance-level concept labels. Unsupervised concepts could reduce manual workload. In this thesis, we aim to reduce the performance gap between unsupervised and supervised concepts for concept-based explanations in computer vision. Targeting the baseline of concept bottleneck models (CBM) with supervised concepts, combined with the advances that unsupervised concepts do not require the concept set designing and labeling, the core contributions in this thesis make the unsupervised concepts an attractive alternative choice for concept-based explanations. Our core contributions are as follows: 1) We propose a new concept learning algorithm, invertible concept-based explanations (ICE). Explanations with unsupervised concepts can be evaluated with fidelity to the original model, like explanations with supervised concepts. Learned concepts are evaluated to be more understandable than baseline unsupervised concept learning methods like k-means clustering methods from ACE; 2) We propose a general framework of concept-based interpretable models with built-in faithful explanations similar to CBM. The framework makes the comparison between supervised and unsupervised concepts available. We show that unsupervised concepts provide competitive performance with model accuracy and concept interpretability; 3) We propose an example of applications using unsupervised concepts with counterfactual explanations, the fast concept-based counterfactual explanations (FCCE). In the ICE concept space, we propose the analytical solution to the counterfactual loss function. The calculation of counterfactual explanations in concept space takes less than 1e-5 seconds. Also, the FCCE is evaluated to be more interpretable through a human survey. In conclusion, previously, unsupervised concepts are not a choice for concept-based explanations as they suffer from many issues, such as being less interpretable and faithful to supervised concept-based explanations like CBM. With all our core contributions, the accuracy and interoperability performance of unsupervised concepts for concept-based explanations is improved to be competitive with supervised concept-based explanations. Since no extra requirements of concept set design and labeling are required, unsupervised concepts are an attractive choice for concept-based explanations in computer vision with competitive performance to supervised concepts. They also bring the benefit that no manual workload of concept set design and labeling is required.
-
ItemExplainable Reinforcement Learning Through a Causal Lens( 2021)This thesis investigates methods for explaining and understanding how and why reinforcement learning agents select actions, from a causal perspective. Understanding the behaviours, decisions and actions exhibited by artificially intelligent agents has been a central theme of interest since the inception of agent research. As systems grow in complexity, the agents' underlying reasoning mechanisms can become opaque and the intelligibility towards humans can be diminished, which can have negative consequences in high-stakes and highly-collaborative domains. The explainable agency of an autonomous agent can aid in transferring the knowledge of this reasoning process to the user to improve intelligibility. If we are to build effective explainable agency, a careful inspection of how humans generate, select and communicate explanations is needed. Explaining the behaviour and actions of sequential decision making reinforcement learning (RL) agents introduces challenges such as handling long-term goals and rewards, in contrast to one-shot explanations in which the attention of explainability literature has largely focused. Taking inspirations from cognitive science and philosophy literature on the nature of explanation, this thesis presents a novel explainable model ---action influence models--- that can generate causal explanations for reinforcement learning agents. A human-centred approach is followed to extend action influence models to handle distal explanations of actions, i.e. explanations that present future causal dependencies. To facilitate an end-to-end explainable agency, an action influence discovery algorithm is proposed to learn the structure of the causal relationships from the RL agent's interactions. Further, a dialogue model is also introduced, that can instantiate the interactions of an explanation dialogue. The original work presented in this thesis reveals how a causal and human-centred approach can bring forth a strong explainable agency in RL agents.