Unification of Smooth and Non-smooth Non-convex Optimization Algorithms

• Applicant: Professor Dr. Peter Ochs
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing ; Mathematics
• Term: from 2018 to 2022
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 397569761

In image processing and machine learning, non-smooth non-convex optimization problems are ubiquitous. For example, such problems naturally arise when certain features are to be explained by a small collection of known representatives. In recent years, significant progress was made by considering optimization problems with a special structure. This led to many new algorithms for non-smooth optimization problems, commonly referred to as splitting methods. However, by specializing to certain structures of the problems, a naturalway to approach minimization problems in optimization was neglected. The actual key isthe approximation quality of certain model functions that are sequentially minimized. In smooth optimization, model functions are usually constructed as Taylor approximations tothe objective function. In non-smooth optimization, there are also "Taylor-like" approximations, which however are not unique. Nevertheless, the approach to define and sequentially minimize non-smooth model functions yields the opportunity to unify several well-known optimization algorithms. As a consequence, results can be transferred from one algorithm to another, and the study of the algorithms is reduced to the essential mathematical structures. The goal of this project is the development of a unifying approach for non-smooth optimization algorithms based on the concept of sequentially minimizing model functions, to provide new convergence guarantees, to develop new algorithms, and thereby explore new fields of applications.

• DFG Programme: Research Grants