Prehensile Interaction: User Interaction Concepts based on Prehensile Hand Behavior
Final Report Abstract
Although interactive 3D environments have gained increasing popularity in recent years, 3D user interfaces still remain rather complex and, in many cases, require special skills and training. Interaction with hand gestures has been an active research area in the 3D user interfaces field, but research from related fields of psychology has rarely been taken into consideration when designing hand-based interfaces. Investigations on reach-to-grasp actions in various domains of psychology have consistently shown that the natural kinematics of prehension allows for predicting the object a human is going to grab and sometimes even the subsequent actions taken on that object. Many studies investigated different factors affecting the prehension, e.g., the object’s size or shape, its specific affordances or intended use, as well as the effect of obstacles and distractors. Some studies also evaluated differences in prehensile behavior when using physical, memorized, or virtual objects. Furthermore, it was shown that the computational complexity of grasp kinematics can be greatly reduced, since the fingers’ dynamics are highly correlated in natural prehension. These insights promise great opportunities for substantially improved interaction in 3D environments, provided that the hand kinematics information is extracted and evaluated on the fly and instantaneously incorporated in the interface. In this project, our objective was to establish a general framework for the analysis of prehensile behavior in the context of human-computer interaction and to explore the applicability of prehensile information to design natural user interfaces for interaction with computer-generated virtual environments. We performed a series of experiments in which the users’ hand kinematics were recorded for different tasks and objects and evaluated with different methods. The aim was to gather sufficient data, develop effective interactive visual methods to analyze it, and examine the benefits and limitations of early prediction of intended objects and tasks based on hand kinematics. The insights gained will allow researchers to further explore the design space for future interactive systems based on natural hand interactions, that is, without requiring special gestures or skills.
Publications
-
Touch Recognition on Complex 3D Printed Surfaces using Filter Response Analysis. 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), 195-200. IEEE.
Valkov, Dimitar; Thiele, Sebastian; Huesmann, Karim; Gebauer, Eike & Risse, Benjamin
-
”Research Topic: Beyond Touch: Free Hand Interaction in Virtual Environments”, Frontiers in Virtual Reality
D. Valkov, R. Teather, F. Daiber, A. U. Batmaz & K. Kim
-
Reach Prediction using Finger Motion Dynamics. Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems, 1-8. ACM.
Valkov, Dimitar; Kockwelp, Pascal; Daiber, Florian & Krüger, Antonio
-
”Grasp Prediction based on Local Finger ¨ Motion Dynamics”
D. Valkov, P. Kockwelp, F. Daiber & A. Kruger
-
”XMTC: Explainable Early Classification of Multivariate Time Series in Reach-to-Grasp Hand Kinematics”
Reyhaneh Sabbagh Gol, Dimitar Valkov & Lars Linsen
