Image-Based Representation of Clothes for Realistic Virtual Try-On

• Applicant: Professor Dr.-Ing. Peter Eisert
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: from 2011 to 2016
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 197264547

One principal intention of computer graphics is the achievement of photorealism. Although modeling, animation and simulation tools for rendering of complex objects (e.g. human bodies, faces, or clothes) have been developed in the last decades, achieving real photorealism by physically simulating material properties and illumination is still computationally demanding and extremely difficult. This is addressed in the current project, where new approaches for image-based rendering and modification of articulated objects with complex appearance properties, such as clothes, have been developed. The methods developed in the current project show very promising results, and we suggest extending the methods so far developed to non-articulated objects, concentrating on the examples of faces. The appearance of clothing as well as faces is very complex as both exhibit fine details, which are very difficult to synthesize. Instead of relying on physical simulation, the developed method, which we call Pose-Space Image-Based Rendering (PS-IBR), uses images as appearance examples to guide complex animation processes, thereby combining the photorealism of images with the ability to animate or modify an object. The main idea of PS-IBR is to define a suitable space (the pose-space) that captures characteristic appearance dependencies of the object, and is used as domain to interpolate and merge images of the object. For articulated objects, such a space can be e.g. parameterized by a skeletal pose representation. New images can then be synthesized given new pose configurations. For the synthesis, the proposed approach combines ideas from example-based animation (pose-space deformation) and image-based rendering. This allows a photorealistic animation and rendering without the need to physically simulate the underlying scene and object properties.Extending the developed pose-space image-based rendering methods to the visualization of faces will allow a photorealistic rendering and synthesis of facial expressions directly from a database of example expression images without complex and computationally demanding simulation of skin bulging, wrinkling effects and reflection properties, as these details are captured by the images. One possible application for the proposed methods is performance-driven facial animation, i.e. the transfer of the facial expressions of one person to another.

• DFG Programme: Research Grants