Hybrid 3D Reconstruction for Geometry-Based Free Viewpoint Video

• Applicant: Professor Dr.-Ing. Thomas Sikora
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: from 2013 to 2016
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 232339705

Our purpose is to develop algorithms for capturing time-varying 3D scene geometry in the context of Free Viewpoint Video applications. This requires a large working volume of at least 2.5m x 2.5m x 3m for capturing a scene. Our strategy involves the generation of a 3D computer graphics model from the captured scene geometry. This model is used at the receiver to reconstruct full parallax Free Viewpoint stereoscopic video. In order to display the time- and view-varying 3D content of the scene on stereoscopic or multi-view displays, it is important that solutions are real-time capable and allow capture of 3D computer graphics models that are as complete and as accurate as possible. Our research strategy attempts to overcome limitations of active and passive techniques for image reconstruction by combining information from both techniques in a hybrid sensor fusion approach. To this end, we employ a multi-projector multi-camera system to project structured light onto the scene and a phase-shifting technique to extract depth and texture information in an active vision approach. The passive multi-camera system is then used to arrive at additional disparity information, which will improve depth and texture reconstruction of the 3D scene model. In order to reduce texture dependent artifacts that stem from large aperture cameras we will derive deconvolution algorithms that use the inverse of an estimated spatially varying point-spread function.

• DFG Programme: Research Grants