Fortschrittliche Lernmethoden für Verfolgung und Detektion im Rahmen einer medizinischen Workflow Analyse

• Antragsteller: Professor Dr. Nassir Navab, Ph.D.
• Mitantragsteller: Professor Dr. Horst Bischof; Professor Dr. Pascal Fua; Professor Dr. Vincent Lepetit
• Fachliche Zuordnung: Bild- und Sprachverarbeitung, Computergraphik und Visualisierung, Human Computer Interaction, Ubiquitous und Wearable Computing
• Förderung: Förderung von 2011 bis 2015
• Projektkennung: Deutsche Forschungsgemeinschaft (DFG) - Projektnummer 179168991

• Erstellungsjahr: 2015

Zusammenfassung der Projektergebnisse

Surgical workflow models are built in order to derive and analyse statistical properties of a surgery for recovering the phase of the operation, staff training, data visualization, report generation and monitoring. Building a workflow model requires sufficient amount of data from different sources and sensors such as medical instruments and imaging devices. In this project, we propose to use a multi-view RGB-camera system that automatically estimates the body pose of the surgeons and medical staff. Our goal is to perform 3D human pose estimation of multiple individuals from multiple views inside the operating room. The body poses compose an additional input signal to the framework of the surgical workflow modelling. To that end, we introduce different human body models for pose estimation in the 2D and 3D space based on RGB image input. Moreover, we present a unique dataset for human pose estimation in the operating room that captures a simulated medical operation using a multi-view camera system. We evaluate our models in standard human pose datasets, as well as in the operating room and demonstrate state-of-the-art performance.

Projektbezogene Publikationen (Auswahl)

• “A user-centered and workflow-aware unified display for the operating room,” in MICCAI Workshop on Modeling and Monitoring of Computer Assisted Interventions (M2CAI), 2012
  R. Stauder, V. Belagiannis, L. Schwarz, A. Bigdelou, E. Soehngen, S. Ilic, and N. Navab
  
• “Segmentation based particle filtering for real-time 2d object tracking,” in Computer Vision–ECCV 2012, pp. 842–855, Springer, 2012
  V. Belagiannis, F. Schubert, N. Navab, and S. Ilic
  
• “3D pictorial structures for multiple human pose estimation,” in CVPR 2014-IEEE International Conference on Computer Vision and Pattern Recognition, 2014
  V. Belagiannis, S. Amin, M. Andriluka, B. Schiele, N. Navab, and S. Ilic
  
• “Holistic human pose estimation with regression forests,” in Articulated Motion and Deformable Objects, pp. 20–30, Springer, 2014
  V. Belagiannis, C. Amann, N. Navab, and S. Ilic
  
• “Multiple human pose estimation with temporally consistent 3D pictorial structures,” in Computer Vision–ECCV 2014, ChaLearn Looking at People Workshop, Springer, 2014
  V. Belagiannis, X. Wang, B. Schiele, P. Fua, S. Ilic, and N. Navab
  
• “Robust optimization for deep regression,” in Computer Vision (ICCV), 2015 IEEE International Conference on, IEEE, 2015
  V. Belagiannis, C. Rupprecht, G. Carneiro, and N. Navab
  
• “3D pictorial structures revisited: Multiple human pose estimation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 10, pp. 1929-1942, 1 Oct. 2016
  V. Belagiannis, S. Amin, M. Andriluka, B. Schiele, N. Navab, and S. Ilic
  
• “Parsing human skeletons in an operating room,” Machine Vision and Applications, October 2016, Volume 27, Issue 7, pp 1035–1046
  V. Belagiannis, X. Wang, H. Beny Ben Shitrit, K. Hashimoto, R. Stauder, Y. Aoki, M. Kranzfelder, A. Schneider, P. Fua, S. Ilic, H. Feussner, and N. Navab