Spatial Modelling and Reasoning

• Applicant: Dr.-Ing. Jan Eric Lenssen
• Subject Area: Methods in Artificial Intelligence and Machine Learning ; Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: since 2025
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 556415750

Humans have powerful abilities to infer the world around them from a limited amount of observations. From just two image streams, for example, they are able to locate themselves in the 3D environment, identify and locate objects, infer possibilities for interaction, and reason about unobserved areas in the vicinity. All these abilities are enabled by previously learned knowledge that humans gather over the course of their lives by observing and interacting with the 3D world. The project described in this research proposal aims to tackle fundamental challenges in machine learning to replicate these capabilities with a computer vision system. Application areas for such a system would be manifold, including areas such as robotics, autonomous driving, and generative 3D modeling. Formally, the given tasks can be categorized as stochastic inverse problems, the problem of inferring uncertain factors that led to a given set of observations, e.g. the 3D structure, semantics, or physical parameters that produced the observed images. We propose to perform research towards two intertwined goals: efficient spatial representations that allow to model the factors involved in the given inverse problems, and machine learning methods that perform spatial reasoning across these representations. For the second aspect, the focus will lie on large-scale generative models, which we believe can be the foundation for large-scale, spatial reasoning models.

• DFG Programme: Emmy Noether Independent Junior Research Groups