DEXMAN: Improving robot’s DEXterous MANipulability by learning stiffness-based human motor skills and visuo-tactile exploration

• Applicants: Qiang Li, Ph.D.; Professor Dr. Jianwei Zhang, since 12/2020
• Subject Area: Automation, Mechatronics, Control Systems, Intelligent Technical Systems, Robotics ; Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: from 2019 to 2023
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 410916101

• Final Report Year: 2023

Final Report Abstract

Roboticists have made huge efforts to mimic the human hand, not only from the form but also from the functionalities. However, robustly grasping and manipulating an unknown object/tool is still an open question to be thoroughly solved. This motivates the present project. We will investigate a human motor skill extraction-based approach to achieve robust dexterous grasping and in-hand manipulation on a robotic arm/hand system. • A novel framework of augmented dynamic movement primitives DMPs embedding perception information for human skill extraction and generalization to new tasks • Reconstructing and tracking an unknown object by exploiting interactive manipulation and multi-modal feedback Roboticists have made huge efforts to mimic the human hand, not only from the form but also from the functionalities. However, robustly grasping and manipulating an unknown object/tool is still an open question to be thoroughly solved. This motivates the present project. We will investigate a human motor skill extraction-based approach to achieve robust dexterous grasping and in-hand manipulation on a robotic arm/hand system. • A novel framework of augmented dynamic movement primitives DMPs embedding perception information for human skill extraction and generalization to new tasks • Reconstructing and tracking an unknown object by exploiting interactive manipulation and multi-modal feedback

Publications

• A Review of Tactile Information: Perception and Action Through Touch. IEEE Transactions on Robotics, 36(6), 1619-1634.
  Li, Qiang; Kroemer, Oliver; Su, Zhe; Veiga, Filipe Fernandes; Kaboli, Mohsen & Ritter, Helge Joachim
  
• Intent inference in shared-control teleoperation system in consideration of user behavior. Complex & Intelligent Systems, 8(4), 2971-2981.
  Wang, Liangliang; Li, Qiang; Lam, James; Wang, Zheng & Zhang, Zhengyou
  
• Learning compliant grasping and manipulation by teleoperation with adaptive force control. 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 717-724. IEEE.
  Zeng, Chao; Li, Shuang; Jiang, Yiming; Li, Qiang; Chen, Zhaopeng; Yang, Chenguang & Zhang, Jianwei
  
• Learning Optimal Impedance Control During Complex 3D Arm Movements. IEEE Robotics and Automation Letters, 6(2), 1248-1255.
  Naceri, Abdeldjallil; Schumacher, Tobias; Li, Qiang; Calinon, Sylvain & Ritter, Helge
  
• Multi-View Merging for Robot Teleoperation With Virtual Reality. IEEE Robotics and Automation Letters, 6(4), 8537-8544.
  Wei, Dong; Huang, Bidan & Li, Qiang
  
• Robot Learning Human Skills and Intelligent Control Design. CRC Press.
  Yang, Chenguang; Zeng, Chao & Zhang, Jianwei
  
• Simultaneously Encoding Movement and sEMG-Based Stiffness for Robotic Skill Learning. IEEE Transactions on Industrial Informatics, 17(2), 1244-1252.
  Zeng, Chao; Yang, Chenguang; Cheng, Hong; Li, Yanan & Dai, Shi-Lu
  
• "Tactile Sensing, Skill Learning and Robotic Dexterous Manipulation", Elsevier, 2, April,2022, eBook ISBN: 9780323904179
  Qiang Li; Shan Luo; Zhaopeng Chen; Chenguang Yang & Jianwei Zhang
  
• "Tactile servoing" IROS2022 Tutorial
  Qiang Li & Nathan Lepora
  
• Editorial: Biological-Inspired Autonomous Mobile Manipulation: Challenges in Design, Control, and Real-World Applications. Frontiers in Neurorobotics, 16.
  Bing, Zhenshan; Chen, Fei; Li, Rui; Zhong, Junpei & Li, Qiang
  
• Explainable Hierarchical Imitation Learning for Robotic Drink Pouring. IEEE Transactions on Automation Science and Engineering, 19(4), 3871-3887.
  Zhang, Dandan; Li, Qiang; Zheng, Yu; Wei, Lei; Zhang, Dongsheng & Zhang, Zhengyou
  
• Generalization of Robot Force-Relevant Skills Through Adapting Compliant Profiles. IEEE Robotics and Automation Letters, 7(2), 1055-1062.
  Zeng, Chao; Li, Shuang; Fang, Bin; Chen, Zhaopeng & Zhang, Jianwei
  
• Guest editorial: Dexterous manipulation. Industrial Robot: the international journal of robotics research and application, 49(4), 601-602.
  Fang, Bin; Li, Qiang; Chen, Fei & Wan, Weiwei
  
• Improving Object Grasp Performance via Transformer-Based Sparse Shape Completion. Journal of Intelligent & Robotic Systems, 104(3).
  Chen, Wenkai; Liang, Hongzhuo; Chen, Zhaopeng; Sun, Fuchun & Zhang, Jianwei
  
• Multi-fingered Tactile Servoing for Grasping Adjustment under Partial Observation. 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE.
  Liu, Hanzhong; Huang, Bidan; Li, Qiang; Zheng, Yu; Ling, Yonggen; Lee, Wangwei; Liu, Yi; Tsai, Ya-Yen & Yang, Chenguang
  
• Robotic dexterous manipulation: from tele-operation to autonomous learning and adaptive control. Complex & Intelligent Systems, 8(4), 2809-2811.
  Li, Qiang; Liu, Chao; Yang, Chenguang; Chen, Fei & Ritter, Helge
  
• Editorial: Advanced learning control in physical interaction tasks. Frontiers in Robotics and AI, 10.
  Zeng, Chao; Guo, Jing; Li, Qiang & Yang, Chenguang
  
• Tactile-Based Object Pose Estimation Employing Extended Kalman Filter. 2023 International Conference on Advanced Robotics and Mechatronics (ICARM), 118-123. IEEE.
  Lin, Qiguang; Yan, Chaojie; Li, Qiang; Ling, Yonggen; Zheng, Yu; Lee, Wangwei; Wan, Zhaoliang; Huang, Bidan & Liu, Xiaofeng