Towards More Realistic Deep Generative Inpainting
Loading...
Citations
Altmetric:
Altmetric::
Associated Records
Authors
File Licence
Editors
Date
2021
Source Title
Journal Title
Journal ISSN
Volume Title
Publisher
Melbourne Authors
Affiliation
Computing and Information Systems
Document Type
PhD thesis
Subject Code
Citation
Access Status
Open Access
DOI
Access URL
Open Access URL
Open Access at PMC
Linked Resource URL
NHMRC Grant Code
ARC Grant Code
Description
© 2021 Ang Li
Abstract
The current era has witnessed an explosion of information, where users are dealing with an overwhelming number of media such as images and videos on their electronic devices every day. Processing and editing these media has become an increasing need in our daily life. Inpainting, one important yet challenging visual editing technique in the field of computer vision, refers to the art of restoring missing regions in a given medium (e.g., a damaged image) based on the information from undamaged regions. Although empirical inpainting techniques have existed for many years, this topic has attracted even more popularity due to the recent development in deep neural networks. Inpainting techniques have a wide range of applications in the real world and are extensively adopted in various scenarios, such as restoration of damaged images/videos, removal of unwanted objects, and loss recovery of image compression transmission.
In this thesis, we focus on designing deep inpainting techniques to achieve more realistic inpainting results. To be specific, four research problems in three different inpainting scenarios are intensively studied. The first research problem aims to maintain semantic consistency from the perspective of model architecture for single image inpainting. We propose an inpainting model that is capable of using semantically interpretable information for image inpainting. The second research problem attempts to maintaining semantic consistency from the perspective of optimization for single image inpainting. We exploit an information geometric measure Local Intrinsic Dimensionality to enforce, in deep feature space, the alignment between the data submanifolds learned by an inpainting model and those of the original data. The third research problem aims to devise one of the first deep learning-based models for stereo image inpainting. We present an Iterative Geometry-Aware Cross Guidance Network, which explicitly learns the geometry correlation and alternately narrows down the missing regions of the two views in an iterative manner. The fourth research problem focuses on addressing the current challenges of video inpainting and designing an improved model. We propose a novel video inpainting network to realize effective context aggregation by exploiting both short-term and long-term reference information.
Considering that deep inpainting techniques have the risk of being manipulated for image forgery, we also address the research problem of deep inpainting detection to combat inpainting-based image forgery. Specifically, we make the first attempt towards universal detection of deep image inpainting, where a detection network trained on one single type of training data can detect a wide range of deep image inpainting methods.
Notes
Terms of Use
Keywords
Computer Vision, Generative Adversarial Network, Image Processing, Inpainting