Improving the understanding of neuromuscular gait control using deep reinforcement learning

• Applicants: Professor Jan Reinhard Peters, Ph.D.; Professor Dr. André Seyfarth
• Subject Area: Automation, Mechatronics, Control Systems, Intelligent Technical Systems, Robotics ; Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: since 2021
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 456562029

Musculoskeletal disorders are one of the leading causes of human disability. The risk of such disabilities is increasing in many countries with an aging population like Germany. Assistive devices (e.g. exoskeletons, prosthesis, etc.) can facilitate human locomotor function and improve walking performance (e.g. balance, metabolic cost, etc.). It has been shown that human walking performance can be improved by the exoskeleton with subject specific control parameters using the human-in-the-loop (HIL) optimization approach. However, the HIL approach is limited in terms of gait diversity (e.g. different walking speeds) and optimization time (e.g. one hour of continuous walking on a treadmill). The large amount of time required to find the optimal parameters using a HIL-like method might not be feasible and/or practical (e.g. for elderly people and patients). A dynamic neuromuscular gait model capable of generating rich human-like locomotion behaviors at kinematic, kinetic and muscle levels can significantly reduce the HIL optimization time by optimizing the parameters first in the simulation and then transferring it to the hardware setup. Therefore in this project, considering the complexity of human neuromuscular control, we propose to develop a deep reinforcement learning (deep-RL) based framework capable of generating rich individual specific human walking behaviors. By simulating musculoskeletal locomotion dynamics, we expect superior predictive capabilities at three levels: (1) individual steady and non-steady gait, (2) response dynamics to unexpected perturbations, and (3) gait assistance dynamics. Here, the learned neural network of the model represents (in a schematic way) the spinal cord neural circuitry mapping sensory inputs to muscle stimulations. We plan to use the individual human rich gait data for our model to learn the walking behaviors across all the three aspects listed above. The quality of the learned gait model will be evaluated in perturbed gait scenarios and assistive gait scenarios using a leg exoskeleton. The proposed deep-RL based framework will not only improve the understanding of human neuromuscular gait control but also aid in developing AI-based gait controllers which could then be transferred to the hardware system with minimal online optimization.

• DFG Programme: Research Grants