Electrical and Electronic Engineering

Spectral imaging captures information in one or more selective bands across the electromagnetic spectrum, permitting the objects in the world to be identified by their absorption or reflection characteristics. Advancements in spectrally selective imaging have primarily been in colour imaging in the visible domain; however, infrared detectors have also enjoyed technological advances that position them ideally for thermal spectral imaging. Advanced spectrally selective imaging systems in longwave infrared (LWIR) thermal wavelengths of 8-14 microns can produce unique thermal fingerprints of objects by recording the heat radiation emitted from objects, thereby creating additional knowledge of the world otherwise difficult to acquire with colour cameras. Therefore, advanced spectral imaging finds important applications in precision agriculture (e.g., early detection of plant diseases), non-invasive medical diagnosis (e.g., vein and dental analysis, skin screening), mining (e.g., non-destructive testing), environmental monitoring (e.g., greenhouse gas detection) and recycling (e.g., plastic classification). However, existing LWIR multi- and hyperspectral imaging systems are expensive and bulky (with cryogenic cooling) and demand time and resources to process several images. Further, LWIR spectral imaging is hindered by the lack of materials responding to thermal wavelengths to design wavelength filters and the low resolution of thermal sensors to design a multi-band filter mosaic compared to their counterpart in the visible wavelengths. Recently, miniaturized infrared spectrometers were reported in the thermal wavelengths. However, they work only with a single isolated object using an active blackbody in the background and fail to detect multiple objects in real scenes. They collect an average emission from multiple objects using single or multiple detectors, which cannot be further resolved due to missing spatial information. There has been an ever-increasing demand for miniaturized and CMOS-compatible LWIR sensors performing imaging spectroscopy to realize their full potential with increased on-chip integration and new compact applications. In this thesis, I design and demonstrate lightweight and high-performance computational infrared imaging technology to enable joint spatial and spectral data acquisition in LWIR wavelengths. I propose and discuss promising solutions for handheld, mass-producible and affordable LWIR multi- and hyperspectral sensing systems using existing monochrome thermal sensors with a focus on plasmonic filters, sensor engineering and artificial intelligence. The first part of this thesis is focused on designing narrowband filter technology towards LWIR multi- and hyperspectral imagers. I begin by presenting optical metasurfaces and designing nano-optical filters with hexagonal lattices of hole/disk geometries to create surface plasmonic resonances in the LWIR regime. I perform comprehensive detector studies and detailed analyses of nano optical filters to accurately tailor the spectral responsivities of the LWIR plasmonic filters for imaging applications. I propose CMOS standard infrared plasmonic filters offering horizontal scalability, narrow spectral width, micron size thickness, and high transmission features. In the second part of this thesis, I explore time-resolved and spatially-resolved multispectral imaging systems for acquiring spatial image information in selective spectral bands. I substantiate the findings from the plasmonic filter simulations by experimentally realizing the novel LWIR plasmonic filters. Their instrumentation is explored by stacking into thermal image sensors through a filter wheel, and by integrating the filter mosaic into the camera to make a compact single-sensor imaging system. I experimentally demonstrate their time- or spatial-multiplexing performance in real-time and recover high-resolution multispectral images with deep imaging. In the third part of this thesis, I develop a deep learning-based LWIR imaging spectroscopy system prototype for acquiring more spectral information with selective spatial images in real time. This is a computational LWIR spectral imaging system acquired by the joint design of a snapshot multispectral imager at the hardware front, and a novel deep learning-based algorithmic spectroscopy concept for rapid spectral reconstruction at the software front. Snapshot images are acquired in selective spectral bands using LWIR plasmonic filters stacked to multiple detectors, which are further processed with deep neural network architecture to rapidly predict the spectra. The power of our deep learning-based imaging spectrometer is experimentally demonstrated by identifying four minerals: amethyst, calcite, pyrite, and quartz. The proposed technique is a simple and approximate 'uncooled LWIR thermal hyperspectral imaging system', which can be used to identify multiple objects by retrieving the spectral fingerprint in a real scene without recording a large number of images and without needing an active blackbody source. I thus demonstrate next-generation thermal sensing systems by merging nanoplasmonic sensors and artificial intelligence. Our results will form the basis for a snapshot, lightweight, compact, and low-cost hyperspectral LWIR imagers enabling diverse applications in chemical detection, precision agriculture, disease diagnosis, environmental sensing and industry vision.

As applications of deep learning continue to be discovered and implemented, the problem of robustness becomes increasingly important. It is well established that deep learning models have a serious vulnerability against adversarial attacks. Malicious attackers targeting learning models can generate so-called "adversarial examples'' that are able to deceive the models. These adversarial examples can be generated from real data by adding small perturbations in specific directions. This thesis focuses on the problem of explaining vulnerability (of neural networks) to adversarial examples, an open problem which has been addressed from various angles in the literature. The problem is approached geometrically, by considering adversarial examples as points which lie close to the decision boundary in a high-dimensional feature space. By invoking results from high-dimensional geometry, it is argued that adversarial robustness is impacted by high data dimensionality. Specifically, an upper bound on robustness which decreases with dimension is derived, subject to a few mathematical assumptions. To test this idea that adversarial robustness is affected by dimensionality, we perform experiments where robustness metrics are compared after training neural network classifiers on various dimension-reduced datasets. We use MNIST and two cognitive radio datasets for our experiments, and we compute the attack-based empirical robustness and attack-agnostic CLEVER score, both of which are approximations of true robustness. These experiments show correlations between adversarial robustness and dimension in certain cases.

Electrical and Electronic Engineering - Theses

Permanent URI for this collection

Filters

Date

Author

Subject

Type

Settings

Sort By

Results per page

Statistics

Citations

Search Results