Robotic-Specific Machine Learning

• Applicant: Professor Dr. Oliver Brock
• Subject Area: Automation, Mechatronics, Control Systems, Intelligent Technical Systems, Robotics ; Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: from 2017 to 2024
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 329426068

• Final Report Year: 2024

Final Report Abstract

In previous work, we showed that using suitable priors significantly improved the learning performance and generalization in robotics. However, this method was limited to fully observable settings, a state representation without semantic structure and simple robotic tasks. The motivation of this project was to develop methods with new priors and extend previous works to more complex tasks. Firstly, we relaxed the assumption on full observability as robots only have partial information about themselves and their environments. We examined this partial observability with high-dimensional input with state estimation problem. Jonschkowski et al. combined machine learning with existing state estimation algorithms, which functioned as algorithmic priors. Compared to machine learning approaches without these algorithmic priors, our method showed a significant reduction in error rates. Secondly, we showed it is possible to learn structured state representations without supervision by utilizing priors that are consistent with the movement of physical objects in the real world. This method, successfully applied to different robotic tasks like inverted pendulum, cart-pole, and ball-in-cup in simulation, yielded state representations closely matching true system states. Thirdly, we extended the learning of representations from a single task to multiple tasks. We developed a method for learning state representations for multiple tasks using robotic priors. The algorithm led to better policies with less data compared to other methods on a simulated multi-task scenario. Lastly, in the extension, we extended the previous findings to complex real-world robotic tasks. We chose the soft sensorized dexterous hand as the experimental platform. The robotic tasks involved shape classification via in-hand manipulation and moving objects on a tabletop. We identified a key difficulty in such systems: the unmodeled nuisance factors related to the complex data generating process. A good state representation should disentangle such factors by utilizing priors and we are still working towards this goal.

Link to the final report

https://doi.org/10.34657/19901

Publications

• PVEs: Position-Velocity Encoders for Unsupervised Learning of Structured State Representations. New Frontiers for Deep Learning in Robotics Workshop at RSS
  Rico Jonschkowski, Roland Hafner, Jonathan Scholz & Martin Riedmiller
  
• Differentiable Particle Filters: End-to-End Learning with Algorithmic Priors. Robotics: Science and Systems XIV. Robotics: Science and Systems Foundation.
  Jonschkowski, Rico; Rastogi, Divyam & Brock, Oliver
  
• Learning Robotic Perception Through Prior Knowledge. PhD Thesis.
  Rico Jonschkowski
  
• State Representation Learning with Robotic Priors for Partially Observable Environments. 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 6693-6699. IEEE.
  Morik, Marco; Rastogi, Divyam; Jonschkowski, Rico & Brock, Oliver
  
• Surprisingly Robust In-Hand Manipulation: An Empirical Study. Robotics: Science and Systems XVII. Robotics: Science and Systems Foundation.
  Bhatt*, Aditya; Sieler*, Adrian; Puhlmann, Steffen & Brock, Oliver