Ein KI-basiertes Mehrzwecksystems um Assistive Robotersysteme zur effektiven Objektmanipulation durch Embodied Teleoperation und Shared Control nutzbar zu machen

• Antragsteller: Professor Jan Reinhard Peters, Ph.D.
• Fachliche Zuordnung: Bild- und Sprachverarbeitung, Computergraphik und Visualisierung, Human Computer Interaction, Ubiquitous und Wearable Computing
• Förderung: Förderung von 2020 bis 2025
• Projektkennung: Deutsche Forschungsgemeinschaft (DFG) - Projektnummer 442430069

• Erstellungsjahr: 2023

Zusammenfassung der Projektergebnisse

Dexterous manipulation of objects is a core task in robotics. Because of the design complexity needed for robot controllers even for simple manipulation tasks, robots currently in use are mostly limited to specific tasks within a known environment. Within the CHIRON project, we aim to develop an AI empowered general purpose robotic system for dexterous manipulation of complex and unknown objects in rapidly changing, dynamic and unpredictable real-world environments. This will be achieved through intuitive embodied robotic teleoperation under optimized shared-control between the human operator enhanced with an intuitive haptic interface and the robot controller empowered with AI-based vision and learning skills. The privileged use case of such a system is assistance for “stick-to-bed” patients or elders with limited physical ability in their daily life object manipulation tasks, e.g., fetching a bottle of water and pouring it into a glass, through an intuitive and embodied robot tele-operated by themself. Such object manipulations would be otherwise not possible for them. To make possible such an embodied tele-operated robotic system for dexterous manipulation, thus without any assumption about the object to be manipulated and the operating environment, the CHIRON project features unique innovations simultaneously on different AI applied domains: robotics, computer vision and machine learning.

Projektbezogene Publikationen (Auswahl)

• Panda-gym : Open-source goal-conditioned environments for robotic learning, NeurIPS 2021 4th Robot Learning Workshop: Self-Supervised and Lifelong Learning, 14 December 2021
  Quentin Gallouédec, Nicolas Cazin, Emmanuel Dellandréa & Liming Chen
  
• Attention Regularized Laplace Graph for Domain Adaptation. IEEE Transactions on Image Processing, 31, 7322-7337.
  Luo, Lingkun; Chen, Liming & Hu, Shiqiang
  
• Auditory Feedback for Enhanced Sense of Agency in Shared Control. Sensors, 22(24), 9779.
  Morita, Tomoya; Zhu, Yaonan; Aoyama, Tadayoshi; Takeuchi, Masaru; Yamamoto, Kento & Hasegawa, Yasuhisa
  
• Cutaneous Feedback Interface for Teleoperated In-Hand Manipulation. 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 605-611. IEEE.
  Zhu, Yaonan; Colan, Jacinto; Aoyama, Tadayoshi & Hasegawa, Yasuhisa
  
• Intension Reading based Task Outcome Prediction for Operability Improvement of Time-Delayed Teleoperation System. In 2022 International Symposium on Micro-Nano Mechatronics and Human Science (MHS), IEEE, 1–1
  Keisuke Fusano, Yaonan Zhu, Jacinto Enrique Colan Zaita, Tadayoshi Aoyama & Yasuhisa Hasegawa
  
• Robust Reinforcement Learning: A Review of Foundations and Recent Advances. Machine Learning and Knowledge Extraction, 4(1), 276-315.
  Moos, Janosch; Hansel, Kay; Abdulsamad, Hany; Stark, Svenja; Clever, Debora & Peters, Jan
  
• A Shared Control Framework for Enhanced Grasping Performance in Teleoperation. IEEE Access, 11, 69204-69215.
  Zhu, Yaonan; Jiang, Bingheng; Chen, Qibin; Aoyama, Tadayoshi & Hasegawa, Yasuhisa
  
• Discriminative Noise Robust Sparse Orthogonal Label Regression-Based Domain Adaptation. International Journal of Computer Vision, 132(1), 161-184.
  Luo, Lingkun; Hu, Shiqiang & Chen, Liming
  
• Hierarchical policy blending as inference for reactive robot control. In 2023 IEEE International Conference on Robotics and Automation (ICRA) (pp. 10181-10188). IEEE
  Kay Hansel, Julan Urain, Jan Peters & Georgia Chalvatzaki
  
• Hierarchical Policy Blending As Optimal Transport. In Learning for Dynamics and Control Conference (pp. 797-812). PMLR
  An Thai Le, Kay Hansel, Jan Peters & Georgia Chalvatzaki
  
• Human Preferences and Robot Constraints Aware Shared Control for Smooth Follower Motion Execution.” In 2023 IEEE International Symposium on Micro-Nano Mechatronics and Human Science (MHS)
  Qibin Chen, Yaonan Zhu, Kay Hansel, Tadayoshi Aoyama & Yasuhisa Hasegawa
  
• Improvement in the Manipulability of Remote Touch Screens Based on Peri-Personal Space Transfer. IEEE Access, 11, 43962-43974.
  Yamamoto, Kento; Zhu, Yaonan; Aoyama, Tadayoshi; Takeuchi, Masaru & Hasegawa, Yasuhisa
  
• Intention-reflected predictive display for operability improvement of time-delayed teleoperation system. ROBOMECH Journal, 10(1).
  Zhu, Yaonan; Fusano, Keisuke; Aoyama, Tadayoshi & Hasegawa, Yasuhisa
  
• LLM-Based Human-Robot Collaboration Framework for Manipulation Tasks.” In 2023 IEEE International Symposium on Micro-Nano Mechatronics and Human Science (MHS)
  Haokun Liu, Yaonan Zhu, Kenji Kato, Izumi Kondo, Tadayoshi Aoyama & Yasuhisa Hasegawa
  
• PickSim: a dynamically configurable Gazebo pipeline for robotic manipulation”, Advancing Robot Manipulation Through Open-Source Ecosystems-2023 IEEE International Conference on Robotics and Automation (ICRA) Conference Workshop
  Guillaume Duret, Nicolas Cazin, Mahmoud Ali, Florence Zara, Emmanuel Dellandréa, Jan Peters & Liming Chen
  
• Visual Tactile Sensor Based Force Estimation for Position-Force Teleoperation. 2022 IEEE International Conference on Cyborg and Bionic Systems (CBS), 49-52. IEEE.
  Zhu, Yaonan; Nazirjonov, Shukrullo; Jiang, Bingheng; Colan, Jacinto; Aoyama, Tadayoshi; Hasegawa, Y.; Belousov, Boris; Hansel, Kay & Peters, Jan
  
• When continual learning meets robotic grasp detection: a novel benchmark on the Jacquard dataset, 18th International Conference on Computer Vision Theory and Applications (VISAPP’2023), Lisbon, Portugal, 19-21 February, 2023
  Rui Yang, Matthieu Grard, Amaury Depierre, Emmanuel Dellandréa & Liming Chen