Foundations of Lifelong Reinforcement Learning

• Applicant: Dr. Claire Vernade, Ph.D.
• Subject Area: Theoretical Computer Science
• Term: since 2022
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 468806714

Lifelong Reinforcement Learning (LRL) is a recently introduced paradigm for machine learning that endows interactive learning agents with the ability to learn high-level skills across tasks. The objective is to learn how to memorize the right information from previous taks and reuse it appropriately on future similar tasks. These ideas are rooted in human learning studies and they interestingly combine the powerful meta-learning and reinforcement learning paradigms. However, the existing research on LRL remains so far mostly empirical and lacks consistent metrics, objectives and a canonical statistical model. Our main research question is: How hard is LRL? Can we design an optimal and computationally efficient agent? What are the canonical learning problems this agent should solve? This project proposes to construct the theoretical Foundations of LRL (FoLiReL) and to contribute to understanding the statistical trade-offs at stake in this long-term exploration-versus-exploitation problem. We will connect meta-learning and RL theories, as well as our recent advances in Multi-Armed Bandits. In particular, we will explore the complexity of the problem under the lens of statistical lower bounds on the achievable metrics. Based on this new notion of optimality, we will lay down the benchmark for future theoretical or empirical research in the field.

• DFG Programme: Independent Junior Research Groups