Stochastic optimal control problems with time delay: Discretisation, numerical solution, and delay sensitivity

• Applicant: Dr. Markus Fischer
• Subject Area: Mathematics
• Term: from 2008 to 2010
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 87508781

An optimal control problem consists in the task of controlling a system, starting from an initial state, over a certain period of time in a “best possible” way by choosing, at each point in time, a control action, i. e. a value for certain parameters of the system. Performance of controls is measured by means of a cost functional. A system in continuous time is usually described by a parametrised differential equation. In the stochastic case, the differential equation possesses a deterministic component as well as a component depending on random quantities. Time delay may occur in the control or in the state dynamics. In the former case, there is a delay either in the implementation of the control actions or in the information flow which underlies the choice of the control actions. In the latter case, the future evolution of the system depends not only on the current state (and on future controls and disturbances), but directly also on the history of the system. Examples of such control problems are given by growth models from biology or economics, certain models of financial markets and by applications in engineering. The aim of our project is to develop, mathematically analyse and implement procedures for the numerical solution of stochastic optimal control problems with time delay, especially with time delay in the state dynamics. In addition, we will study and quantify the effect of small delays in the control. The long-term objective is to provide a theoretical basis as well as tools for the numerical treatment of control problems with delay.

• DFG Programme: Research Fellowships
• International Connection: USA
• Host: Professor Dr. Paul Dupuis