Algorithm Control: Efficient Learning to Control Algorithm Parameters
Final Report Abstract
It is widely known that the performance of many algorithms depends on their hyperparameters. To avoid tedious and error-prone manual tuning, automated approaches have been developed to efficiently achieve peak algorithm performance. However, these approaches have the drawback of setting hyperparameter values only once, even though many hand-designed internal heuristics of AI algorithms are typically reactive in nature, adjusting parameters dynamically during runtime. In this project, we developed approaches to automatically learn such reactive, dynamic heuristics for hyperparameter control from data and thus further enhance the efficiency of algorithms. To achieve this, we leveraged the latest advances in reinforcement learning by framing the task of algorithm control as a contextual Markov Decision Process (MDP). This allowed us to make algorithms across several domains significantly more efficient, such as in automated planning, evolutionary algorithms, and deep learning. To enable the research community to continue working dynamically in these domains, we also created a freely available benchmark library. Additionally, we developed methods to adapt reinforcement learning for algorithm control by explicitly modeling the temporal component and instance context. This project, therefore, provided the theoretical, practical and empirical foundations and the first benchmark for dynamic algorithm configuration, demonstrated the potential of reinforcement learning for this task for the first time, extended reinforcement learning through the lens of algorithm control, and thus significantly improved algorithms across various domains.
Publications
-
Dynamic Algorithm Configuration: Foundation of a New Meta-Algorithmic Framework. Frontiers in Artificial Intelligence and Applications. IOS Press.
Biedenkapp, André; Bozkurt, H. Furkan; Eimer, Theresa; Hutter, Frank & Lindauer, Marius
-
Verfahren, Vorrichtung UND Computerprogramm zum Einstellen eines Hyperparameters. IPC Nr. G06N 7/ 00 A I. P. N. EP3748551
M. Lindauer et al.
-
DACBench: A Benchmark Library for Dynamic Algorithm Configuration. Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, 1668-1674. International Joint Conferences on Artificial Intelligence Organization.
Eimer, Theresa; Biedenkapp, André; Reimer, Maximilian; Adriansen, Steven; Hutter, Frank & Lindauer, Marius
-
DEVICE AND METHOD FOR PLANNING AN OPERATION OF A TECHNICAL SYSTEM. IPC Nr. G06N 5/ 00 A I. P. N. US2021383245
J. Spitz et al.
-
Learning Heuristic Selection with Dynamic Algorithm Configuration. Proceedings of the International Conference on Automated Planning and Scheduling, 31, 597-605.
Speck, David; Biedenkapp, André; Hutter, Frank; Mattmüller, Robert & Lindauer, Marius
-
Self-Paced Context Evaluation for Contextual Reinforcement Learning. In: Proceedings of the 38th International Conference on Machine Learning (ICML’21). Ed. by M. Meila and T. Zhang. Vol. 139
T. Eimer et al.
-
TempoRL: Learning When to Act. In: Proceedings of the 38th International Conference on Machine Learning (ICML’21). Ed. by M. Meila and T. Zhang. Vol. 139. Proceedings of Machine Learning Research. PMLR, 2021, pp. 914–924
A. Biedenkapp et al.
-
Vorrichtung und Verfahren zur Planung eines Betriebs eines technischen Systems. IPC Nr. G06Q 50/ 04 A I. P. N. DE102020207114
J. Spitz et al.
-
Automated Dynamic Algorithm Configuration. Journal of Artificial Intelligence Research, 75, 1633-1699.
Adriaensen, Steven; Biedenkapp, André; Shala, Gresa; Awad, Noor; Eimer, Theresa; Lindauer, Marius & Hutter, Frank
-
Method and device for learning a strategy and for implementing the strategy. IPC Nr. G06N 3/ 12 A I. P. N. US2022027743
S. Adriaenssen et al.
-
PI is back! Switching Acquisition Functions in Bayesian Optimization. In: NeurIPS Workshop on Gaussian Processes, Spatiotemporal Modeling, and Decision-making Systems. Ed. by T. Broderick et al. 2022
C. Benjamins et al.
-
Towards Automated Design of Bayesian Optimization via Exploratory Landscape Analysis. In: NeurIPS 2022 Workshop on Meta-Learning. Ed. by F. Ferreira et al. 2022
C. Benjamins et al.
-
Verfahren und Vorrichtung zum Lernen einer Strategie und Betreiben der Strategie. IPC Nr. G06N 3/ 08 A I. P. N. DE102020209281
S. Adriaenssen et al.
-
Self-Adjusting Weighted Expected Improvement for Bayesian Optimization. In: Proceedings of the Second International Conference on Automated Machine Learning. Ed. by A. Faust et al. Proceedings of Machine Learning Research, 2023
C. Benjamins et al.
-
Instance Selection for Dynamic Algorithm Configuration with Reinforcement Learning: Improving Generalization. Proceedings of the Genetic and Evolutionary Computation Conference Companion, 563-566. ACM.
Benjamins, Carolin; Cenikj, Gjorgjina; Nikolikj, Ana; Mohan, Aditya; Eftimov, Tome & Lindauer, Marius
-
Verfahren zum Trainieren eines Algorithmus des maschinellen Lernens durch ein bestärkendes Lernverfahren. Deutsches Patent- und Markenamt (DPMA) IPC Nr. G06N20/00. P. N. DE102022210480A1
T. Eimer et al.
