The Epistemology of Statistical Learning Theory

• Applicant: Dr. Tom Sterkenburg
• Subject Area: Theoretical Philosophy ; Theoretical Computer Science
• Term: from 2019 to 2023
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 437206810

Machine learning has an ever increasing impact on science and society as a whole. This calls for a persistent effort towards a better understanding, not only of the (societal, ethical, and otherwise) consequences of the ubiquitous use of machine learning methods, but also of these methods themselves. What explains their apparent success? In what sense and to what extent do they lead to reliable conclusions? What are their unavoidable pitfalls and limitations? These questions have to do with the nature and justification of uncertain or inductive inference: fundamental epistemological questions. Nevertheless, philosophers have to date engaged little with these questions in relation to our currently most prominent methods of inductive inference, those given by modern machine learning algorithms. At the same time, researchers in the theory of machine learning have developed powerful mathematical frameworks to examine the functioning of learning algorithms. Statistical learning theory (SLT), in particular, constitutes the theoretical framework underlying much of modern machine learning. The aim of my project is to employ the framework of SLT to investigate philosophical problems of inductive inference, and to embed these investigations in a wider epistemological appraisal of machine learning methods. This main aim finds concrete shape in a set of objectives within three stages. The first stage concerns the fundamental limitations of inductive inference, and specifically, in the context of SLT, the no-free-lunch theorems as expressions of these. I will work out a unified perspective by arguing for a most fruitful interpretation of the no-free-lunch theorems, and by giving an account of how results within philosophy can be understood as instantiations of the no-free-lunch theorems. The second stage rather concerns the success of learning methods, and possible explanations thereof: specifically the adherence to a simplicity preference in inductive inference, the principle of Occam's razor. I will spell out a minimal yet genuine sense in which SLT provides an argument for the justification of Occam’s razor.The first two stages come together, finally, in the current scientific debate on how to explain the practical success of the particular learning paradigm of deep neural nets. In the third stage I will analyze this debate as an important case study in the philosophy of inductive inference. This debate, moreover, ties in with a more general dicussion on the role of machine learning theory and its implications for traditional epistemology. I will argue against the persistent view that the practical success of machine learning warrants an epistemology that downplays the role of theoretical analysis. This also serves as a defense of the working assumption of my project that the theory of machine learning is invaluable for its epistemological appraisal.

• DFG Programme: Research Grants