Probabilistic Numerics - Probabilistic Programming for Autonomous Systems

• Applicant: Professor Dr. Philipp Hennig
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing ; Mathematics
• Term: from 2015 to 2017
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 265399621

Probabilistic machine learning requires numerical operations, but existing numerical methods cannot deal with probabilistic inputs. The proposed research will establish a suite of algorithms for elementary problems in linear algebra, optimization, and the solution of differential equations, which take probability distributions as inputs and return probability distributions that capture the error of the numerical computation. Together, these algorithms will provide a paradigm for the propagation of uncertainty through the high-level computational pipeline of autonomous machines.Underlying the research program is a fundamental observation, that numerical methods can themselves be interpreted as autonomous inference algorithms, because they estimate not directly accessible quantities (matrix decompositions, extrema, solutions to differential equations) from observations (gradients, function values) at actively chosen locations. In fact, existing, highly popular numerical procedures for the above settings, such as the method of conjugate gradients, the BFGS algorithm, and Runge-Kutta methods can be re-interpreted more or less directly as probabilistic inference methods. Doing so typically raises additional degrees of freedom, which control the method's error estimates and are not part of the classic analogues. Rules for fixing these free parameters are a main aim of the research programme.The proposed research has two complementary aims. Three separate work streams will study probabilistic methods for linear algebra, nonlinear optimization, and the solution of ordinary differential equations, respectively, working toward a clearer mathematical understanding and improved functionality within uncertain settings. This will give practitioners more flexibility in the use of numerical methods for the individual tasks, an outcome whose scientific, societal and economic value is potentially substantial but, due to the systemic nature of numerical algorithms, difficult to quantify. A fourth work stream will tie the results together into a coherent suite of methods, and study their use for the propagation of uncertainty through the computational pipeline of a high-level autonomous machine. The propagated error measure offers a novel diagnostic quantity that both human designers and the machine itself can use to improve performance and better control computational cost.

• DFG Programme: Independent Junior Research Groups