Electrical and Electronic Engineering - Research Publications
Permanent URI for this collection
Search Results
1 - 3 of 3
-
ItemNo Preview AvailableAn Information-Theoretic Analysis for Transfer Learning: Error Bounds and Applications( 2022-07-12)Transfer learning, or domain adaptation, is concerned with machine learning problems in which training and testing data come from possibly different probability distributions. In this work, we give an information-theoretic analysis on the generalization error and excess risk of transfer learning algorithms, following a line of work initiated by Russo and Xu. Our results suggest, perhaps as expected, that the Kullback-Leibler (KL) divergence D(μ||μ′) plays an important role in the characterizations where μ and μ′ denote the distribution of the training data and the testing test, respectively. Specifically, we provide generalization error upper bounds for the empirical risk minimization (ERM) algorithm where data from both distributions are available in the training phase. We further apply the analysis to approximated ERM methods such as the Gibbs algorithm and the stochastic gradient descent method. We then generalize the mutual information bound with ϕ-divergence and Wasserstein distance. These generalizations lead to tighter bounds and can handle the case when μ is not absolutely continuous with respect to μ′. Furthermore, we apply a new set of techniques to obtain an alternative upper bound which gives a fast (and optimal) learning rate for some learning problems. Finally, inspired by the derived bounds, we propose the InfoBoost algorithm in which the importance weights for source and target data are adjusted adaptively in accordance to information measures. The empirical results show the effectiveness of the proposed algorithm.
-
ItemTracking and regret bounds for online zeroth-order Euclidean and Riemannian optimisation( 2020-10-01)We study numerical optimisation algorithms that use zeroth-order information to minimise time-varying geodesically-convex cost functions on Riemannian manifolds. In the Euclidean setting, zeroth-order algorithms have received a lot of attention in both the time-varying and time-invariant cases. However, the extension to Riemannian manifolds is much less developed. We focus on Hadamard manifolds, which are a special class of Riemannian manifolds with global nonpositive curvature that offer convenient grounds for the generalisation of convexity notions. Specifically, we derive bounds on the expected instantaneous tracking error, and we provide algorithm parameter values that minimise the algorithm’s performance. Our results illustrate how the manifold geometry in terms of the sectional curvature affects these bounds. Additionally, we provide dynamic regret bounds for this online optimisation setting. To the best of our knowledge, these are the first regret bounds even for the Euclidean version of the problem. Lastly, via numerical simulations, we demonstrate the applicability of our algorithm on an online Karcher mean problem.
-
ItemModelling of synaptic interactions between two brainstem half-centre oscillators that coordinate breathing and swallowing( 2021)Abstract Respiration and swallowing are vital orofacial motor behaviours that require the coordination of the activity of two brainstem central pattern generators (r-CPG, sw-CPG). Here, we use computational modelling to further elucidate the neural substrate for breathing-swallowing coordination. We progressively construct several computational models of the breathing-swallowing circuit, starting from two interacting half-centre oscillators for each CPG. The models are based exclusively on neuronal nodes with spike-frequency adaptation, having a parsimonious description of intrinsic properties. These basic models undergo a stepwise integration of synaptic connectivity between central sensory relay, sw- and r-CPG neuron populations to match experimental data obtained in a perfused brainstem preparation. In the model, stimulation of the superior laryngeal nerve (SLN, 10s) reliably triggers sequential swallowing with concomitant glottal closure and suppression of inspiratory activity, consistent with the motor pattern in experimental data. Short SLN stimulation (100ms) evokes single swallows and respiratory phase resetting yielding similar experimental and computational phase response curves. Subsequent phase space analysis of model dynamics provides further understanding of SLN-mediated respiratory phase resetting. Consistent with experiments, numerical circuit-busting simulations show that deletion of ponto-medullary synaptic interactions triggers apneusis and eliminates glottal closure during sequential swallowing. Additionally, systematic variations of the synaptic strengths of distinct network connections predict vulnerable network connections that can mediate clinically relevant breathing-swallowing disorders observed in the elderly and patients with neurodegenerative disease. Thus, the present model provides novel insights that can guide future experiments and the development of efficient treatments for prevalent breathing-swallowing disorders. Key points The coordination of breathing and swallowing depends on synaptic interactions between two functionally distinct central pattern generators (CPGs) in the dorsal and ventral brainstem. We model both CPGs as half-centre oscillators with spike-frequency adaptation to identify the minimal connectivity sufficient to mediate physiologic breathing-swallowing interactions. The resultant computational model(s) can generate sequential swallowing patterns including concomitant glottal closure during simulated 10s stimulation of the superior laryngeal nerve (SLN) consistent with experimental data. In silico, short (100 ms) SLN stimulation triggers a single swallow which modulates the respiratory cycle duration consistent with experimental recordings. By varying the synaptic connectivity strengths between the two CPGs and the sensory relay neurons, and by inhibiting specific nodes of the network, the model predicts vulnerable network connections that may mediate clinically relevant breathing-swallowing disorders.