Localized Statistical Learning with Kernels

• Applicants: Professor Dr. Andreas Christmann; Professor Dr. Ingo Steinwart
• Subject Area: Mathematics
• Term: from 2016 to 2022
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 317622002

• Final Report Year: 2023

Final Report Abstract

The main goal of the project was to investigate whether localized kernel based methods offer comparable or even better results on universal consistency, convergence rates, and statistical robustness than standard kernel based methods, but need much less computation time and computer memory for large data sets. This turned out to be true such that these kernel based methods can be applied now to much larger data sets than before. However, it was not possible to give final answers to every research question on this topic and more research is needed. On the other hand, some interesting questions that were not included in the proposal were positively answered.

Publications

• Total stability of kernel methods. Neurocomputing, 289, 101-118.
  Christmann, Andreas; Xiang, Daohong & Zhou, Ding-Xuan
  
• Universal consistency and robustness of localized support vector machines. Neurocomputing, 315 (2018, 11), 96-106.
  Dumpert, Florian & Christmann, Andreas
  
• Beating SGD Saturation with Tail-Averaging and Minibatching, Proceedings of Neural Information Processing Systems (NeurIPS) 32, 2019
  N. Mücke, G. Neu, and L. Rosasco
  
• Global Minima of DNNs: The Plenty Pantry
  N. Mücke and I. Steinwart
  
• Reducing training time by efficient localized kernel regression, Proceedings of Machine Learning Research, vol. 89 (22nd AISTATS), pp. 2603–2610, 2019
  N. Mücke
  
• Reproducing kernel Hilbert spaces on manifolds: Sobolev and diffusion spaces. Analysis and Applications, 19(03), 363-396.
  De Vito, Ernesto; Mücke, Nicole & Rosasco, Lorenzo
  
• Adaptive learning rates for support vector machines working on data with low intrinsic dimension. The Annals of Statistics, 49(6).
  Hamm, Thomas & Steinwart, Ingo
  
• Intrinsic Dimension Adaptive Partitioning for Kernel Methods. SIAM Journal on Mathematics of Data Science, 4(2), 721-749.
  Hamm, Thomas & Steinwart, Ingo
  
• On the Robustness of Kernel-Based Pairwise Learning. Artificial Intelligence, Big Data and Data Science in Statistics (2022), 111-153. American Geophysical Union (AGU).
  Gensler, Patrick & Christmann, Andreas
  
• Total stability of SVMs and localized SVMs. Journal of Machine Learning Research, vol. 23, pp. 1–41, 2022
  H. K¨hler and A. Christmann