Human Visual Perception and Classification of Materials
Final Report Abstract
Different materials, such as corduroy, gold and leather have distinctive visual appearances due to their meso-scale geometry and the ways they interact with light. Human observers are remarkably adept at recognizing materials across a wide range of viewing conditions, and can be exquisitely sensitive to subtle differences in material appearance, allowing us, for example, to distinguish between fresh and stale food. Because of this, materials pose a major challenge both to perceptual psychology (how does the brain achieve this robust material recognition?) and to computer graphics (how can material appearance be simulated in a truly convincing way?). Despite the importance of the topic, at the start of the project, very little indeed was known about the perception of material properties. Additionally, the combination of perception and computer graphics was also a relatively new idea, with only a few well-known labs routinely using psychophysical methods to test and develop computer graphics algorithms. In the project, we used a combination of computer graphics, image analysis and psychophysics to advance our understanding of the visual perception of materials. In turn, we also used the perceptual insights that we gained, to advance computer graphics methods for portraying materials. The main research questions were: 1) What image cues does the visual system use to infer material appearance? 2) How can knowledge of these cues be exploited to make simulations of material appearance more convincing or more efficient? 3) What are the perceptual parameters relating different materials to one another? 4) How can knowledge of these parameters be used to improve the creation of novel material appearances along intuitive dimensions? We have made substantial advances in our understanding of all of these questions, although there remain, of course, many salient questions for future research. We started by using (existing) parametric models of surfaces, which model the physics of glossy, transparent and translucent objects. We used these models to simulate a continuous range of different materials, such as different types of glass and plastic. We then used the images for two purposes: first to study the statistical properties of the images; and second, to test human perception of the different materials. Using multidimensional scaling, maximum likelihood difference scaling, asymmetric matching and factor analysis of subjective ratings, we measured how changes in the images affect the apparent properties of the materials. In so doing we gained insight into the cues and perceptual dimensions of transparent, translucent and glossy surfaces. We also applied such techniques to empirically measured BTF samples collected by our collaborators at Bonn University. As a result of the experiments we have the following main advances in our understanding of: 1) the cues involved in recognizing translucent and transparent materials (spatial cues, contrast, color, etc.), 2) how motion cues contribute to the recognition of glossy surfaces; 3) how parametric changes in surface appearance affect the appearance and aesthetic qualities of materials; 4) how lighting affects the perception of gloss, leather textures and 3D shape; 5) how these observations can be exploited to simulate the appearance of different materials, allowing us to digitally edit the appearance of materials in photographs. During the funded period, there has been a significant increase in interest in material perception, and in the use of perception to facilitate computer graphics. We have played a central role in this development. We have organized a number of international symposia on both these topics and published a textbook entitled: “Visual Perception from a Computer Graphics Perspective”. Important future questions emerging from the project include the following. How do cues other than the optical properties of surfaces affect the estimation of materials (e.g. the way materials move and change shape)? How are the cues that we have identified measured and represented in the human brain? How are perceptual estimates of illumination and shape related to estimates of surface materials? As a result of the project and additional collaborations, we are currently preparing a submission to the EU for a Marie-Curie Initial Training Network on the perception of shape, materials and illumination, which aims to answer some of these questions.
Publications
- 2006. Image-Based Material Editing. ACM Transactions on Graphics (Proceedings of SIGGRAPH 06), Vol. 25. 2006, Issue 3, pp. 654-663.
Kahn, E. A., Reinhard, E., Fleming, R. W., Bülthoff, H. H.
(See online at https://dx.doi.org/10.1145/1141911.1141937) - 2006. Sketching Shiny Surfaces: 3D Shape Extraction and Depiction of Specular Surfaces. ACM Transactions on Applied Perception,
Vol. 3. 2006, Issue 3, pp. 262-285.
Weidenbacher, U., Bayerl, P., Neumann, H., R. W. Fleming
(See online at https://dx.doi.org/10.1145/1166087.1166094) - 2007. Distortion in 3D shape estimation with changes in illumination. Proceedings of the 4th Symposium on Applied Perception in Graphics and Visualization (APGV 2007), pp. 99-105.
Caniard, F. and R. W. Fleming
- 2007. Do HDR displays support LDR content?: a psychophysical evaluation. ACM Transactions on Graphics (Proceedings of SIGGRAPH 07), Vol. 26. 2007, Issue 3, Article No. 38.
Akyüz, A. O., Fleming, R. W., Riecke, B. E., Reinhard, E., H. H. Bülthoff
(See online at https://dx.doi.org/10.1145/1276377.1276425) - 2007. Perception and prediction of simple object interactions. Proceedings of the 4th Symposium on Applied Perception in Graphics and Visualization (APGV 2007), pp. 27-34.
Nusseck, M., Lagarde, J., Bardy, B., Fleming, R. W., H. H. Bülthoff
- 2007. Perceptual reparameterization of material properties. Proceedings of the International Symposium on Computational Aesthetics in Graphics, Visualization, and Imaging (CAe‘07), pp. 89-96
Cunningham, D. W., Wallraven, C., Fleming, R. W. and W. Strasser
(See online at https://dx.doi.org/10.1016/j.visres.2013.11.004) - 2009. Evaluation of reverse tone mapping through varying exposure conditions. ACM Transactions on Graphics, Vol. 28. 2009, Issue 5, Article No. 160.
Masia B, Agustin S, Fleming RW, Sorkine O and Gutierrez D
(See online at https://dx.doi.org/10.1145/1618452.1618506) - 2009. Image statistics for clustering paintings according to their visual appearance In: Computational Aesthetics 2009, Fifth International Symposium on Computational Aesthetics in Graphics,
Visualization, and Imaging, Eurographics, Aire-La-Ville, Switzerland, 2009, pp. 1-8.
Spehr M., Wallraven C. and Fleming R.W.
(See online at https://dx.doi.org/10.2312/COMPAESTH/COMPAESTH09/057-064) - Three dimensional shape and the perception of Physical stability. Journal of Vision, Vol.9. 2009, Issue 8: 47.
Fleming, R.W., M. Singh
(See online at https://dx.doi.org/10.1167/9.8.47) - 2011. Perceived Object Stability Depends on Multisensory Estimates of Gravity. PLoS One, Vol. 6. 2011, Issue 4: e19289.
Barnett-Cowan M., Fleming R.W., Singh M. and Bülthoff H.H.
(See online at https://dx.doi.org/10.1371/journal.pone.0019289) - 2011. Visual Perception from a Computer Graphics Perspective. CRC Press, Wellesley, MA, USA.
Thompson, W. B., Fleming, R. W. Creem-Regehr, S. and J. Stefanucci
(See online at https://dx.doi.org/10.1201/b10927-25) - 2011. Visual Perception of Thick Transparent Materials.
Psychological Science, vol. 22. 2011, no. 6, pp. 812-820.
Fleming R.W., Jäkel F. and L.T. Maloney
(See online at https://dx.doi.org/10.1177/0956797611408734)
