Comprehensive Human Performance Capture from Monocular Video Footage

• Applicants: Professor Dr.-Ing. Marcus Magnor; Professor Dr.-Ing. Bodo Rosenhahn
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: from 2015 to 2021
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 265624298

Goal of the proposed research is to comprehensively, photo-realistically model human performances from conventional, monocular video footage. We plan to devise methods to estimate the geometry, pose, motion, illumination as well as texture and reflectance characteristics of a person from a single video sequence. Furthermore, we will develop algorithms to augment texture details, to edit pose and motion, and to convincingly render the model into novel scenes. We intend to employ model- as well as example-based approaches from Computer Vision in conjunction with interactive correction, augmentation, and error-concealing rendering methods from Computer Graphics. Our joint research shall make contributions to the general challenge of underconstrained reconstruction for photo-realistic modeling and rendering. The project intends to lay the methodical foundations for creating digital doubles of actors based on nothing more than their movies, enabling digital conservation of even already deceased actors.

• DFG Programme: Research Grants