Engineering
http://hdl.handle.net/11343/345
Mon, 18 Feb 2019 18:00:14 GMT
20190218T18:00:14Z

Highly efficient distributed hypergraph analysis: realtime partitioning and quantized learning
http://hdl.handle.net/11343/220744
Highly efficient distributed hypergraph analysis: realtime partitioning and quantized learning
Jiang, Wenkai
Hypergraphs have been shown to be highly effective when modeling a wide range of applications where highorder relationships are of interest, such as social network analysis and object classification via hypergraph embedding. Applying deep learning techniques on large scale hypergraphs is challenging due to the size and complex structure of hypergraphs. This thesis addresses two problems of hypergraph analysis, realtime partitioning and quantized neural networks training, in a distributed computing environment.
When processing a large scale hypergraph in realtime and in a distributed fashion, the quality of hypergraph partitioning has a significant influence on communication overhead and workload balance among the machines participating in the distributed processing. The main challenge of realtime hypergraph partitioning is that hypergraphs are represented as a dynamic hypergraph stream formed by a sequence of hyperedge insertions and deletions, where the structure of a hypergraph is constantly changing. The existing methods that require all information of a hypergraph are inapplicable in this case as only a subgraph is available to the algorithm at a time. We solve this problem by proposing a streaming refinement partitioning (SRP) algorithm that partitions a realtime hypergraph flow in two phases. With extensive experiments on a scalable hypergraph framework named HyperX, we show that SRP can yield partitions that are of the same quality as that achieved by offline partitioning algorithms in terms of communication overhead and workload balance.
For machine learning tasks over hypergraphs, studies have shown that using deep neural networks (DNNs) can improve the learning outcomes. This is because the learning objectives in hypergraph analysis are becoming more complex these days, where features are difficult to define and are highlycorrelated. DNNs can be used as a powerful classifier to construct features automatically. However, DNNs require high computational power and network bandwidth as the size of DNN models are getting larger. Moreover, the widely adopted training algorithm, stochastic gradient descent (SGD), suffers in two main problems: vast communication overhead that comes from the broadcasts of parameters during the partial gradient aggregations, and the inherent variance between partial gradients, making the training process even longer as it impedes the convergence rate of SGD. We investigate these two problems in depth. Without sacrificing the performance, we develop a quantization technique to reduce the communication overhead and a new training paradigm, named cooperated lowprecision training (CLPT), in which importance sampling is used to reduce variance, and the master and workers collaborate together to make compensation for the precision loss due to the quantization.
Incorporating deep learning techniques into distributed hypergraph analysis shows a great potential in query processing and knowledge mining on highdimensional data records where relationships among them are highly correlated. On one hand, such a process takes the advantage of strong representational power of DNNs as an appearancebased classifier; on the other hand, such a process exploits hypergraph representations to gain benefits from its strong capability in capturing highorder relationships.
© 2018 Wenkai Jiang
Mon, 01 Jan 2018 00:00:00 GMT
http://hdl.handle.net/11343/220744
20180101T00:00:00Z

Autoregressive generative models and multitask learning with convolutional neural networks
http://hdl.handle.net/11343/220740
Autoregressive generative models and multitask learning with convolutional neural networks
Schimbinschi, Florin
At a high level, sequence modelling problems are of the form where the model aims to predict the next element of a sequence based on neighbouring items. Common types of applications include timeseries forecasting, language modelling, machine translation and more recently, adversarial learning. One main characteristic of such models is that they assume that there is an underlying learnable structure behind the data generation process, such as it is for language. Therefore, the models used have to go beyond traditional linear or discrete hidden state models.
Convolutional Neural Networks (CNNs) are the de facto state of the art in computer vision. Conversely, for sequence modelling and multitask learning (MTL) problems, the most common choice are Recurrent Neural Networks (RNNs). In this thesis I show that causal CNNs can be successfully and efficiently used for a broad range of sequence modelling and multitask learning problems. This is supported by applying CNNs to two very different domains, which highlight their flexibility and performance: 1) traffic forecasting in the context of highly dynamic road conditions with nonstationary data and normal granularity (sampling rate) and a high spatial volume of related tasks; 2) learning musical instrument synthesisers with stationary data and a very high granularity (high sampling rate raw waveforms) and thus a high temporal volume, and conditional side information.
In the first case, the challenge is to leverage the complex interactions between tasks while keeping the streaming (online) forecasting process tractable and robust to faults and changes (adding or removing tasks). In the second case, the problem is highly related to language modelling, although much more difficult since, unlike words, multiple musical notes can be played at the same time, therefore making the task much more challenging.
With the ascent of the Internet of Things (IoT) and Big Data becoming more common, new challenges arise. The four V‘s of Big Data (Volume, Velocity, Variety and Veracity) are studied in the context of multitask learning for spatiotemporal (ST) prediction problems.
These aspects are studied in the first part of this thesis. Traditionally such problems are addressed with static, nonmodular linear models that do not leverage Big Data. I discuss what the four V‘s imply for multitask ST problems and finally show how CNNs can be set up as efficient classifiers for such problems, if the quantization is properly set up for nonstationary data.
While the first part is predominantly datacentric, focused on aspects such as Volume (is it useful?) and Veracity (how to deal with missing data?) the second part of the thesis addresses the Velocity and Variety challenges. I also show that even for prediction problems set up as regression, causal CNNs are still the best performing model as compared to state of the art algorithms such as SVRS and more traditional methods such as ARIMA. I introduce TRUVAR (Topologically Regularized Universal Vector AutoRegression) which, as I show, is a robust, versatile realtime multitask forecasting framework which leverages domainspecific knowledge (task topology), the Variety (task diversity) and Velocity (online training).
Finally, the last part of this thesis is focused on generative CNN models. The main contribution is the SynthNet architecture which is the first capable of learning musical instrument synthesisers endtoend. The architecture is derived by following a parsimonious approach (reducing complexity) and via an indepth analysis of the learned representations of the baseline architectures. I show that the 2D projection of each layer gram activations can correspond to resonating frequencies (which gives each musical instrument it‘s timbre). SynthNet trains much faster and it’s generation accuracy is much higher than the baselines.
The generated waveforms are almost identical to the ground truth. This has implications in other domains where the the goal is to generate data with similar properties as the data generation process (i.e. adversarial examples).
In summary, this thesis makes contributions towards multitask spatiotemporal time series problems with causal CNNs (set up as both classification and regression) and generative CNN models. The achievements of this thesis are supported by publications which contain an extensive set of experiments and theoretical foundations.
© 2018 Dr Florin Schimbinschi
Mon, 01 Jan 2018 00:00:00 GMT
http://hdl.handle.net/11343/220740
20180101T00:00:00Z

Neural tissue electrical modelling at micro and macro scales
http://hdl.handle.net/11343/220735
Neural tissue electrical modelling at micro and macro scales
Sergeev, Evgeni Nikitich
A better understanding of electrical stimulation of the retina by neural prostheses may be essential for progress to be made towards a viable massmarket design of such devices. Dividing the problem into electrode models, target neuron models, and models of the tissue filling the volume between the electrodes and neurons, we focus on the tissue models.
Prior work suggests that to model the relevant tissue, the neural retina, a standard, homogeneous, volume conductor may not be an appropriately faithful choice, even one with an anisotropic conductivity and permittivity. This is due to the capacitance of neural membranes and the macroscopic dimensions of the cablelike neural processes forming the tissue. Prior work on the subject resulted in alternative models being proposed (meanfield models). However, while those prior models may be solved approximately, there had been no wellestablished method to estimate the amount of error in those approximate solutions. We propose an alternative approach to derive a meanfield model, on the basis of finite element discretisations of a reference microstructural model. The latter is made up of infinitelylong axons running parallel to one another. To estimate the accuracy of those finite element solutions, we adapt the Global Convergence Index (GCI) technique. Our adaptation incorporates roundoff error into the GCI technique in a systematic and conservative way. Our resulting meanfield model, the quantifieduncertainty (QU) model, produces solutions together with uncertainty estimates. While there are some differences between the QU model and prior models, they produce compatible solutions, in the sense that solutions using the prior models generally fall within the uncertainty band of solutions produced using the QU model, under boundary conditions of practical relevance. We describe a detailed method for solving a simple instance of a situated application problem incorporating the QU model.
The derivation of the QU model proceeds by transforming the microstructural model into an appropriate spectral domain, then solving for a point source in a large, coarselydiscretised instance, in order to establish the claim that the farfield behaviour in two lattice directions is sufficient to characterise the whole response. We then solve finelymeshed finite element models corresponding to these two directions, under farfield boundary conditions. Observing that the solutions converge exponentially (and rapidly) towards functions with useful symmetry properties, we take advantage of the latter to constrain equivalent discrete models, reduced so as to represent only the quantities relevant to the meanfield description: potential, current flow across the fibres, and current flow along the fibres ("absorption"). We find equivalent continuousdomain models to the discrete models. We were also able to express the QU model in terms of two rational interpolating functions with a small number of coefficients. The uncertainty part of the QU model is formed so as to cover the differences between the two directions mentioned above, in addition to accounting for the fitting residuals from interpolation, for the discretisation error from the finite element representation and for the roundoff error from solving the finite element matrices, including their conditioning.
© 2018 Dr Evgeni Nikitich Sergeev
Mon, 01 Jan 2018 00:00:00 GMT
http://hdl.handle.net/11343/220735
20180101T00:00:00Z

MOFmediated destruction of cancer using Fenton reaction
http://hdl.handle.net/11343/220711
MOFmediated destruction of cancer using Fenton reaction
Ranjiburachaloo, Hadi
Cancer which is the second greatest cause of death worldwide has reached critical levels. In the past various therapies including photodynamic, photothermal and chemotherapy are utilized for selective tumor treatment. Unfortunately, these methods suffer from various problems which limit their efficiency and performance. For this reason, novel strategies are being explored which improve the efficiency of these traditional therapeutic methods or treat the tumor cells directly. One such strategy utilizing the Fenton reaction has been investigated by many groups for the possible treatment of cancer cells. This therapy involves the utilisation of existing high levels of H2O2 in cancer cells to react with iron nanoparticles following the Fenton reaction to produce hydroxyl radicals capable of killing the cells. However, studies which attempted to use classical Fenton reaction alone to destroy the tumor cells, requires high concentrations of nanoparticles in order to be toxic to cancer cells. For this reason, there has not seen a successful nanoparticle which can treat cancer cells using the Fenton reaction without the need for external H2O2 sources.
The aim of my work was to synthesize and develop novel metal organic frameworks (MOFs) for cancer treatment using the Fenton reaction. These specific nanoparticles can be utilized directly to destroy the cancer cells via the Fenton reaction or indirectly to deliver the Fenton reagent into cancer cells. In the first approach, a novel reduced iron metalorganic framework nanoparticle with cytotoxicity specific to cancer cells was fabricated. Iron present on the MOF can react with high levels of hydrogen peroxide found specifically in cancer cells to increase the hydroxyl radical concentration. The hydroxyl radicals oxidize proteins, lipids and/or DNA within the biological system to decrease cell viability. In vitro experiments demonstrate that this novel nanoparticle is cytotoxic to cancer cells through generation of hydroxyl radical using the cell’s own hydrogen peroxide. However, this emerging method is largely restricted due to the poor selectivity of reported nanoparticles. Subsequent improvements in nanoparticle size were facilitated by PEGylation on the particles through surfaceinitiated atom transfer radical polymerization, thus improving the stability, reducing the size and increasing the selectivity. In vitro experiments show that the selectivity index increased from 2.45 to 4.48 for HeLa cells, which is significantly higher than those reported in the literature for similar strategies.
Finally, in an alternative approach, pHresponsive MOFs have been utilized for hemoglobin (Fenton reagent) and glucose oxidase (starvation reagent) delivery into the cancer cells. In a slightly acidic environment of cancer cells, GOx is released and consumes glucose and molecular oxygen that are essential survival nutrients in cancer cells and produces gluconic acid and hydrogen peroxide, respectively. The produced gluconic acid increases the acidity of the tumor microenvironment so completes MOFs destruction and enhances hemoglobin and GOx release. Fe ion from the heme groups of hemoglobin also releases in the presence of both endogenous and produced H2O2 and generate hydroxyl radical. In vitro experiments demonstrate that this novel nanoparticle is cytotoxic to both cancer (HeLa and MCF7) cells at very low concentration (>2 µg/mL).
Due to the great potential of the reported metalorganic frameworks in this thesis, these interesting particles may function as a new type of agents for controlled delivery and hydroxyl radical generation to treat cancer cells
© 2018 Dr. Hadi Ranjiburachaloo
Mon, 01 Jan 2018 00:00:00 GMT
http://hdl.handle.net/11343/220711
20180101T00:00:00Z