Servicenavigation

Project Details

Back

Projekt Print View

Improving Spatial Perception for Medical Augmented Reality with Interactable Depth Layers

Subject Area Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
Software Engineering and Programming Languages
Term from 2016 to 2022
Project identifier Deutsche Forschungsgemeinschaft (DFG) - Project number 290703152
 
Final Report Year 2024

Final Report Abstract

Incorrect spatial interpretation is still one of the most common perceptional problems in medical augmented reality. To further investigate this challenge, this follow-up project worked on new visualization and interaction methods to improve the spatial perception for medical AR. We primarily used projector-based output modalities without requiring special displays such as monitors or hand-held devices, but also expanded our research focus to optical see-through head-mounted displays because of latest developments in that area. Our project was thus focused on the exploration of medical 3D-data in projected and optical see-through AR while aiming at providing additional depth information. To this end, we implemented and compared different interaction methods for the exploration of a projected volumetric image data set. Results of two experiments revealed advantages of using gestures based on the natural motion of hand-held devices to explore volumetric image data in projector-based AR. It was also found that voice commands are a beneficial tool to allow for discrete interaction tasks. In addition, results of two experiments examining depth perception in projector-based AR suggest that enabling the depth cue stereopsis has a great effect on depth estimation accuracy. However, encoding distance information to the appearance of virtual objects may be almost equally contributing. In addition, preserving parts of the actual projection surface occluding underlying virtual structures is beneficial by providing further depth information. Regarding the application area of needle-based interventions, we identified differences between projector-based and optical see-through AR. In general, both methods appear to be equally suitable in this domain. However, the choice of AR display has repercussions on the selected visualization approaches. Projector-based AR may be best implemented using 2D visualizations not requiring stereoscopy. At the same time, optical see-through AR appears more advantageous for 3D contents. With these findings, this project contributes to the general field of medical and projector-based AR. Evidence-based results can improve the interaction with and visualization of AR contents in future applications.

Publications

 
 

Additional Information

© 2025 DFG Disclaimer / Copyright / Privacy Policy

Textvergrößerung und Kontrastanpassung