Verbal instructing of sensor-based robots (VerbBot)
Final Report Abstract
Developing an intuitive robot system has long been the subject of research. Approaches based on graphical user interfaces, voice control, gesture recognition or a combination of these approaches have been tested for various applications. Meanwhile, rudimentary voice control of mobile robots is even possible in the home. In addition to domestic use, however, robotic arms should also be used in small and medium-sized enterprises in the future and, in the best case, be operable there by nonexperts in the field of robotics. One reason why voice control of robotic arms is not yet widespread, apart from the high cost of robotic arms, is the difficulty of mapping natural language instructions into minstructions that the robotic system can understand. Especially for force-based motions, a simplification of the instructions is necessary since the user would not be expected to explicitly specify all parameters necessary for the motion. To overcome this, several new concepts were introduced and investigated in this project, which allow intuitive language-based instruction of force-based robot motions. In this project, preliminary work was first extended that allows mapping of instructions to robot motion based on physical laws and properties by allowing combinations of these motions. To relieve users of the burden of having to specify numerical force values, a model was presented that allows mapping from fuzzy parameters to sharp force parameters. The need for such a model was demonstrated with the help of a special user study (Wizard-of-Botz). Since people tend to omit obvious parameters in instructions, an approach is presented which allows validation of such instructions by means of so-called affordances. Since even syntactically and semantically correct instructions can lead to incorrect executions, for example because the user is not fully aware of the capabilities of the robot arm, an approach is also presented which first executes the movements in a physics-based simulation and checks for possible deviations. The overall system was finally evaluated with the help of a prototype, which implements the above-mentioned approaches. The scientific results were published in five peer-reviewed publications at relevant international conferences and in a dissertation. In addition, the project was reported in a professional journal.
Publications
-
Grounding Verbs for Tool-Dependent, Sensor-Based Robot Tasks. 2018 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 378-383. IEEE.
Wolfel, Kim & Henrich, Dominik
-
Grounding of Uncertain Force Parameters in Spoken Robot Commands. Advances in Intelligent Systems and Computing, 194-201. Springer International Publishing.
Wölfel, Kim & Henrich, Dominik
-
„Gesagt, (gefragt,) getan“, Zeitschrift handling, Nr. 6, 2019
Wölfel K. & Henrich D.
-
Affordance Based Disambiguation and Validation in Human-Robot Dialogue. Annals of Scientific Society for Assembly, Handling and Industrial Robotics, 307–317.
Wölfel, Kim & Henrich, Dominik
-
Simulation-Based Validation of Robot Commands for Force-Based Robot Motions. Lecture Notes in Computer Science, 334-340. Springer International Publishing.
Wölfel, Kim & Henrich, Dominik
-
Wizard of Botz: A Novel Approach to the Wizard of Oz Experiment. Mechanisms and Machine Science, 557-564. Springer International Publishing.
Wölfel, Kim & Henrich, Dominik
