Electrical and Electronic Engineering - Research Publications

Permanent URI for this collection

http://hdl.handle.net/11343/362

Filters

2014 2
Reset filters

Settings
Statistics
Citations
Start date: 2014 × End date: 2014 × Type: Conference Paper × Author: Alpcan, Tansu ×

Search Results

Now showing 1 - 2 of 2
  • Item
    Thumbnail Image
    Information metrics for model selection in function estimation
    Alpcan, T (IEEE, 2014-01-01)
    A model selection framework is presented for function estimation under limited information, where only a small set of (noisy) data points are available for inferring the nonconvex unknown function of interest. The framework introduces information-theoretic metrics which quantify model complexity and are used in a multi-objective formulation of the function estimation problem. The intricate relationship between information obtained through observations and model complexity is investigated. The framework is applied to the hyperparameter selection problem in Gaussian Process Regression. As a result of its generality, the framework introduced is applicable to a variety of settings and practical problems with information limitations such as channel estimation, black-box optimisation, and dual control.
  • Item
    Thumbnail Image
    Can we measure the difficulty of an optimization problem?
    Alpcan, T ; Everitt, T ; Hutter, M (IEEE, 2014)
    Can we measure the difficulty of an optimization problem? Although optimization plays a crucial role in modern science and technology, a formal framework that puts problems and solution algorithms into a broader context has not been established. This paper presents a conceptual approach which gives a positive answer to the question for a broad class of optimization problems. Adopting an information and computational perspective, the proposed framework builds upon Shannon and algorithmic information theories. As a starting point, a concrete model and definition of optimization problems is provided. Then, a formal definition of optimization difficulty is introduced which builds upon algorithmic information theory. Following an initial analysis, lower and upper bounds on optimization difficulty are established. One of the upper-bounds is closely related to Shannon information theory and black-box optimization. Finally, various computational issues and future research directions are discussed.