Online Scene Reconstruction and Understanding

• Applicant: Professor Dr. Leif Kobbelt
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: from 2018 to 2021
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 392037563

• Final Report Year: 2022

Final Report Abstract

In this project a number of novel algorithms have been developed, that efficiently and reliably reconstruct high-quality 3D models from raw point clouds. We broadly explored the use of model-based as well as data-driven methods. For the 3D reconstruction from (laser) scanned point clouds we employed probabilistic models and multi-sensor fusion to make maximum use of all available input information and to guarantee minimum mismatch and noise in the output. This applies to both, shape and texture information. We developed neural network architectures for the creation of new shapes of certain categories and subject to various constraints prescribed by the user. We finally combined a data-driven and a model-based approach in order to generate polygon meshes and layouts on unstructured freeform shapes that are aligned to shape features in a meaningful way.

Publications

• Learning to Reconstruct High-Quality 3D Shapes with Cascaded Fully Convolutional Networks. Lecture Notes in Computer Science, 626-643. Springer International Publishing.
  Cao, Yan-Pei; Liu, Zheng-Ning; Kuang, Zheng-Fei; Kobbelt, Leif & Hu, Shi-Min
  
• Real-time High-accuracy Three-Dimensional Reconstruction with Consumer RGB-D Cameras. ACM Transactions on Graphics, 37(5), 1-16.
  Cao, Yan-Pei; Kobbelt, Leif & Hu, Shi-Min
  
• A Convolutional Decoder for Point Clouds using Adaptive Instance Normalization. Computer Graphics Forum, 38(5), 99-108.
  Lim, Isaak; Ibing, Moritz & Kobbelt, Leif
  
• HeteroFusion: Dense Scene Reconstruction Integrating Multi-Sensors. IEEE Transactions on Visualization and Computer Graphics, 26(11), 3217-3230.
  Yang, Sheng; Li, Beichen; Liu, Minghua; Lai, Yu-Kun; Kobbelt, Leif & Hu, Shi-Min
  
• 3D Shape Generation with Grid-based Implicit Functions. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 13554-13563. IEEE.
  Ibing, Moritz; Lim, Isaak & Kobbelt, Leif
  
• High-Quality Textured 3D Shape Reconstruction with Cascaded Fully Convolutional Networks. IEEE Transactions on Visualization and Computer Graphics, 27(1), 83-97.
  Liu, Zheng-Ning; Cao, Yan-Pei; Kuang, Zheng-Fei; Kobbelt, Leif & Hu, Shi-Min
  
• Learning Direction Fields for Quad Mesh Generation. Computer Graphics Forum, 40(5), 181-191.
  Dielen, Alexander; Lim, Isaak; Lyon, Max & Kobbelt, Leif