RESEARCH:ESTIMATION

Estimation of Reflectance and Illumination for 3D Relighting

Tianli Yu, Hongcheng Wang, Narendra Ahuja and Wei-Chao Chen

INTRODUCTION

The images of a scene under varying illuminations and from different viewpoints are highly interrelated, which makes it possible to predict the object's appearance from new viewpoints or under different illuminations. Most research require known or controllable illumination, which is an impractical assumption for many scenes such as outdoors. Other research that relinquish this constraint often impose other restrictions, such as the uniform specular albedo assumption. In this research, we demonstrate for the first time that observations of the specular appearance from multiple viewpoints are sufficient to solve for illumination, diffuse and specular albedo maps simultaneously up to a scaling ambiguity. Our input images are taken with sparsely distributed viewpoints, in contrast to dense sampling required by many image based rendering methods.

PUBLICATIONS

1. Tianli Yu, Hongcheng Wang, Narendra Ahuja, Wei-Chao Chen, Sparse Lumigraph Relighting by Illumination and Reflectance Estimation from Multi-View Images, Eurographics Symposium on Rendering (

EGSR), 2006

Full Text: PDF (1.7M)2. Tianli Yu, Hongcheng Wang, Narendra Ahuja, Wei-Chao Chen, Sparse Lumigraph Relighting by Illumination and Reflectance Estimation from Multi-View Images, Technical Sketch,

SIGGRAPH, 2006

Full Text: PDF (343k)

ALGORITHM

We formulate the estimation problem as the decomposition of the observed 3D radiance tensor

Rinto three componentsAx_{1}Bx_{2}H, whereB,H, andAare the albedo matrix, illumination matrix, and the light transport tensor (LTT), respectively. LTTAtransforms the contribution of each illumination component to each surface point and is fully specified by the object geometry and a base material's BRDF. This tensor also encodes the non-linearfactors including rotation of directions to the surface normal coordinates and self-shadowing. Given the LTTs, our problem is reduced to solving the albedo matrixBand illumination matrixA. To further constraint the solution, we assume directional environment lighting with no inter-reflection. We then solve this bilinear system by iteratively fixing one set of parameters (HorB) and solving a linear least squares problem for the other. Because LTT is sensitive to reflection directions, we also optimize the bump maps along the process.

DEMO VIDEO

Video: demo (.avi, DiVx 6.2 compressed, ~70MB)If you can not play the video, please download DivX 6.2 codec

Estimation:

input diffuse map specular map 3D shape illumination

Relighting:

changing lighting

changing texture

object insertion

Updated: May.21, 2006