Sim-to-real transfer instrument for artificial intelligence-driven autonomous robots

• Subject Area: Systems Engineering
• Term: Funded in 2025
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 560257124

For its current and future research moving towards robotic artificial general intelligence (AGI) systems, the newly formed Automatic Control Group (RAT) requires a simulation-to-reality transfer submillimeter precision instrument to collect data for training novel artificial intelligence (AI) models and to validate and transfer them from simulation and reality (sim-to-real transfer learning). Robotic AGI-driven autonomy will revolutionize various domains in the coming decades by solving complex problems and generating new capabilities on demand. Autonomous ground and aerial robots have already been successfully used for inspection, maintenance, package delivery, cleaning, and disinfection. The RAT group seeks to push the boundaries of (multiagent) robotic systems by leveraging the current state-of-the-art autonomy level towards smarter systems that will learn and interact with their environment, collaborate with humans, plan their future actions, and execute tasks accurately with the help of AI; grounded in strong foundations from modern control theory and stochastic approximation. The requested instrument is a high precision motion capture ground-truth camera system, completed by a GPU-based workstation and mobile sensory units to offer diversity in sensor and camera angle modality. The instrumentation enables high quality data collection and ground-truth generation, validation, and testing in simulation and real-time. In addition, it can verify robots’ actions to identify limitations of the developed AI-driven navigation methods. The submillimeter precision ground-truth camera instrument is, therefore, critical for thorough tests of autonomous mobile robots at different scales. Further, it allows tracking of dynamic objects even at high speeds and varying illumination conditions, thus providing realtime actions and states of robots. The whole high-precision motion capture camera environment can thus function as one extended sensory system to tackle sim-to-reality transfer problems for the training and real-time deployment of AI-driven navigation methods for higher levels of autonomy. With the sim-to-real transfer learning capability of the motion-capture camera environment, the requested instrument will enable the RAT group to tackle open problems in autonomous robotics and will support several current projects as well as various projects in the coming decades.

• DFG Programme: Major Research Instrumentation
• Major Instrumentation: Instrument zur Übertragung von sim2real für autonome Roboter basierend auf künstlicher Intelligenz
• Instrumentation Group: 3490 Sonstige medizinische Registriergeräte und Zubehör
• Applicant Institution: Universität Paderborn
• Leader: Professor Dr. Erdal Kayacan