Nicht-glatte Bi-level Optimierung in Computer Vision und Machine Learning
Bild- und Sprachverarbeitung, Computergraphik und Visualisierung, Human Computer Interaction, Ubiquitous und Wearable Computing
Zusammenfassung der Projektergebnisse
Most of the models for solving practical problems in Computer Vision, Machine Learning and related Natural Sciences depend on a choice of parameters. Often, the automatic, data based estimation of parameters is a tremendous challenge. This holds, in particular, for Computer Vision and Machine Learning applications with their distinct characteristics of high dimensional data, high dimensional parameter spaces and the fact that problems are naturally modeled with non-smooth functions. Formally, the parameter optimization problem belongs to the class of bi-level optimization problems. Such problems are difficult to solve and require numerical solutions even for low dimensional problems. The goal of this project is the development of a theoretical and practical framework for efficiently solving bi-level optimization problems with the characteristics pointed out above. This will improve the solutions in practical problems, allow us to solve new problems, and will provide theoretical convergence guarantees. Moreover, we expect theoretical insights into heuristic solution strategies in related areas, for example, “backpropagation strategy” (chain rule) for non-differentiable ReLU activation functions in the training of neural networks.
Projektbezogene Publikationen (Auswahl)
-
Automatic Differentiation of Some First-Order Methods in Parametric Optimization. International Conference on Artificial Intelligence and Statistics, 2020
S. Mehmood & P. Ochs
-
Differentiating the Value Function by using Convex Duality. International Conference on Artificial Intelligence and Statistics, 2021
S. Mehmood & P. Ochs
-
Fixed-Point Automatic Differentiation of Forward–Backward Splitting Algorithms for Partly Smooth Functions. Technical Report
S. Mehmood & P. Ochs
-
Automatic Differentiation of Optimization Algorithms with Time-Varying Updates. Technical Report
S. Mehmood & P. Ochs
