Semantic-Aware Augmented Reality Interaction for 3D Robot Workspace Restriction

• Applicant: Professor Dr.-Ing. Matthias König
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: since 2022
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 465783762

The proposed project addresses research of novel 3D semantic-aware interaction methods based on augmented reality techniques in the domain of 3D robot workspace restriction.Mobile robots enter human-centered indoor environments, such as offices or home environments, which results in human-robot shared spaces. The workspaces of such mobile robots, i.e. the space that can be reached by the robots using their locomotion system in 2D/3D, is typically only limited by physical borders, such as walls or furniture. They can operate in the entire (indoor) environment without further restrictions. However, humans sometimes prefer to or need to restrict these workspaces of their mobile robots.These restriction spaces cannot be recognized by a mobile robot on its own because the spaces often do not feature distinct visual characteristics, which could allow their recognition using on-board sensors such as cameras.Therefore, an interaction process between human and mobile robot is necessary. Such an interaction process has to allow the transfer of complex spatial 3D information about a restriction space between human and robot and has to provide complex feedback to the human, e.g. visualization of 3D restriction spaces. Thus, the proposed project deals with research for interactive restriction of a mobile robot’s 3D workspace by non-expert users in order to change the mobile robot’s navigational behavior. Semantically context-aware AR interaction methods will be further developed focusing on the domain of 3D robot workspace restriction. It is the goal to design new 3D interaction methods based on AR, that incorporate semantic context information during user interaction, and to extend the knowledge of interactive 3D robot workspace restriction. This aims at improving 3D user interaction with respect to user requirements. For this purpose, the applicant will build on techniques from the fields of mobile AR, semantic scene understanding and robot workspace restriction. This will lead to novel 3D interaction methods for robot workspace restriction, but that can also be applied to other domains dealing with 3D interaction in general, especially tasks of selection and manipulation. Due to the role of humans as intended users of the proposed interaction methods, the applicant will pursue a user-centered design approach throughout the project to incorporate requirements and feedback of the users in all phases of the project.

• DFG Programme: Research Grants
• International Connection: Denmark