Autonomous Street Crossing with City Navigation Robots
Final Report Abstract
Mobile outdoor robots have the potential to revolutionize the way we live, work, and move around our cities. With their advanced capabilities, mobile robots can perform a variety of tasks, from delivering people and goods, and monitoring traffic to cleaning streets and inspecting buildings. However, urban environments pose enormous challenges for city navigation robots. Urban environments are characterized by complex topologies, dense traffic, and a multitude of moving agents. Therefore, autonomous ground robots operating in urban environments must face immense challenges. They may encounter and interact with different types of agents such as pedestrians, bicyclists, cars, buses, or trucks. What is more, the close and frequent interactions with human agents in urban environments require a careful assessment of the scene and its possible future states in order to minimize the risk of harming human agents or destroying goods. One of the most challenging problems in this context is the task of autonomous street crossing. To cross streets safely and efficiently, autonomous ground robots need to be aware of their surroundings. Understanding one’s surroundings includes on one hand being aware of the static features such as the location of roads, sidewalks, and traffic managing infrastructure (traffic lights and traffic signs). On the other hand, it involves detecting dynamic objects such as moving vehicles, pedestrians, and obstacles that must be detected with high accuracy. Safe and efficient planning of future paths through such urban environments is particularly challenging as not only the current state of the environment must be detected, but also a prediction about future states of the environment must be made and these predictions must be included in the robot’s planning of its future path. In this project, we use the task of autonomous street crossing as a challenging real-world benchmark task for mobile robots in order to evaluate the fidelity of our developed methods. In multiple sub-tasks related to autonomous street crossing, we evaluated current state-of-theart, developed novel ideas, and implemented them in real-world systems. In multiple tasks, we were able to extend the state-of-the-art and gather new insights relevant to any autonomously operating outdoor robot. To summarize some of our findings, we propose novel methods for self-supervised learning and multi-modal learning in the context of autonomous navigation. Concretely, we propose novel methods for robust and data-efficient perception in urban environments, including novel methods for semantic segmentation and obstacle detection, novel methods for lane graph estimation, and novel methods for autonomous vehicle detection and tracking. Furthermore, we identify lane graph estimation as a key enabler for robots operating in urban environments. We present multiple novel approaches that aim at solving lane graph estimation and enable autonomous robots to reason about the abstract topological structure of roads and road intersections.
Publications
-
HeatNet: Bridging the Day-Night Domain Gap in Semantic Segmentation with Thermal Images. 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 8461-8468. IEEE.
Vertens, Johan; Zurn, Jannik & Burgard, Wolfram
-
Lane Graph Estimation for Scene Understanding in Urban Driving. IEEE Robotics and Automation Letters, 6(4), 8615-8622.
Zürn, Jannik; Vertens, Johan & Burgard, Wolfram
-
Self-Supervised Visual Terrain Classification From Unsupervised Acoustic Feature Learning. IEEE Transactions on Robotics, 37(2), 466-481.
Zürn, Jannik; Burgard, Wolfram & Valada, Abhinav
-
Self-Supervised Moving Vehicle Detection From Audio-Visual Cues. IEEE Robotics and Automation Letters, 7(3), 7415-7422.
Zürn, Jannik & Burgard, Wolfram
-
TrackletMapper: Ground Surface Segmentation and Mapping from Traffic Participant Trajectories. In: 6th Annual Conference on Robot Learning. 2022
Jannik Zürn*, Sebastian Weber* & Wolfram Burgard
-
Learning and Aggregating Lane Graphs for Urban Automated Driving. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 13415-13424. IEEE.
Büchner, Martin; Zürn, Jannik; Todoran, Ion-George; Valada, Abhinav & Burgard, Wolfram
-
AutoGraph: Predicting Lane Graphs From Traffic Observations. IEEE Robotics and Automation Letters, 9(1), 73-80.
Zürn, Jannik; Posner, Ingmar & Burgard, Wolfram
