Hybrid 3D Reconstruction for Geometry-Based Free Viewpoint Video
Final Report Abstract
The goal of the project was to improve an active vision system for on-line capturing of 3D textured wire-grid models for TV and cinema applications. The initial hardware setup featured two high-speed monochrome cameras, projectors as well as color cameras. The target distance to the scene objects was between 3-5 meters and covered a volume of 2x2x2 meters. The project’s objectives were to arrive at a three-dimensional reconstruction of the scene that is as complete as possible and as accurate as possible. For the first objective, in addition to active also passive reconstruction methods were used to improve the completeness of the reconstruction using a sensor fusion approach. The active reconstruction method employed a phase shifted structured light system. The existence of calibrated color cameras within this setup allowed for texturing the acquired 3D models. Passive matching methods were utilized to complete the 3D reconstruction. This was achieved by applying a modified hierarchical plane-sweeping approach. The above sensor fusion approach still suffered from inaccurate matching results outside the focal plane of the cameras. Objects with a higher distance to the focal planes appeared blurred in the respective cameras, especially in images from the monochrome high-speed cameras used in the phase measuring triangulation. This “blur” resulted in inaccurate matching of the objects and therefore led to suboptimal accuracy of the 3D reconstruction. We modeled “blur” by a virtual Point Spread Function and a de-convolution filter was designed to improve reconstruction. The algorithms and systems developed in the project enable a much more complete and accurate 3D reconstruction of the scene compared to previous implementations and known competing system.
Publications
- "Consensus-Based Multiview Texturing and Depth-Map Completion," 20th IEEE International Conference on Image Processing, Sep. 2013
Kai Ide, Ivo Keller, and Thomas Sikora
(See online at https://dx.doi.org/10.1109/ICIP.2013.6738438) - "Hybrid 3D Reconstruction for Geometry-Based Free Viewpoint Video," Communication Systems Group, Technische Universität Berlin, Berlin, Dissertation, 2015
Kai Ide
- "Image Guided Phase Unwrapping for real-time 3D-Scanning," 31st IEEE Picture Coding Symposium, May 2015
Thilo Borgmann, Michael Tok and Thomas Sikora
(See online at https://dx.doi.org/10.1109/PCS.2015.7170058)
