Project Details
Projekt
Active learning for stochastic economic model predictive control - nonlinear optimization formulations and numerical methods
Applicant
Professor Dr. Moritz Diehl
Subject Area
Automation, Mechatronics, Control Systems, Intelligent Technical Systems, Robotics
Term
since 2025
Project identifier
Deutsche Forschungsgemeinschaft (DFG) - Project number 535860958
As part of the research unit "Active Learning for Systems and Control (ALeSCo) - Data Informativity, Uncertainty, and Guarantees", this project considers the formulation and approximative solution of stochastic optimal control problems as a means of implicitly defining data informativity for nonlinear dynamic systems with economic objective functions. While nonlinear model predictive control (NMPC) relies on a deterministic model to predict the future system behavior and to derive control decisions based on these predictions, stochastic optimal control aims at explicitly accounting for uncertainties due to stochasticity inherent in the system behavior, modeling errors or estimation uncertainty. However, the resulting stochastic optimal control problems - in theory solved via the stochastic Bellman equation in belief space - are in general intractable to solve due to Bellman's well-known "curse of dimensionality". The aim of this project is to develop approximate, computationally tractable formulations as well as tailored numerical methods for the online solution of stochastic optimal control formulations with nonlinear constrained models and economic costs. In particular, we consider systems whose state cannot be measured directly but needs to be inferred from noisy measurements. These problems are modeled in terms of partially observable Markov decision processes (POMDP) in continuous state and action spaces where a belief of the current system state and system parameters - a Bayesian state estimate - is propagated. The prediction models under consideration might be black-box or grey-box models and shall directly encode a measure of prediction uncertainty - an example is the covariance matrix prediction in an extended Kalman filter. Given that the prediction uncertainty typically has a negative impact on the economic objective, but can potentially be reduced by a smart choice of the chosen control actions, the exact solution of the stochastic optimal control problem will automatically encode a form of active learning strategy that might be classified as implicit dual control. While it is intractable to compute the exact solution, we aim at online optimization based model predictive control formulations that qualitatively preserve the dual control effect. The developed approaches will be based on structured nonlinear programming (NLP) formulations that will be solved in real time by tailored numerical methods to be applicable in practical control applications such as robotics or energy systems.
DFG Programme
Research Units
