RESEARCH:ESTIMATION

 

Estimation of Reflectance and Illumination for 3D Relighting

 

Tianli Yu, Hongcheng Wang, Narendra Ahuja and Wei-Chao Chen

 

 

INTRODUCTION


The images of a scene under varying illuminations and from different viewpoints are highly interrelated, which makes it possible to predict the object's appearance from new viewpoints or under different illuminations. Most research require known or controllable illumination, which is an impractical assumption for many scenes such as outdoors. Other research that relinquish this constraint often impose other restrictions, such as the uniform specular albedo assumption. In this research, we demonstrate for the first time that observations of the specular appearance from multiple viewpoints are sufficient to solve for illumination, diffuse and specular albedo maps simultaneously up to a scaling ambiguity. Our input images are taken with sparsely distributed viewpoints, in contrast to dense sampling required by many image based rendering methods.

 

 

PUBLICATIONS


1. Tianli Yu, Hongcheng Wang, Narendra Ahuja, Wei-Chao Chen, Sparse Lumigraph Relighting by Illumination and Reflectance Estimation from Multi-View Images, Eurographics Symposium on Rendering (EGSR), 2006

Full Text: PDF (1.7M)

2. Tianli Yu, Hongcheng Wang, Narendra Ahuja, Wei-Chao Chen, Sparse Lumigraph Relighting by Illumination and Reflectance Estimation from Multi-View Images, Technical Sketch, SIGGRAPH, 2006  

Full Text: PDF (343k)

 

ALGORITHM


 

 

We formulate the estimation problem as the decomposition of the observed 3D radiance tensor R into three components Ax1Bx2H, where B, H, and A are the albedo matrix, illumination matrix, and the light transport tensor (LTT), respectively. LTT A transforms the contribution of each illumination component to each surface point and is fully specified by the object geometry and a base material's BRDF. This tensor also encodes the non-linear factors including rotation of directions to the surface normal coordinates and self-shadowing. Given the LTTs, our problem is reduced to solving the albedo matrix B and illumination matrix A. To further constraint the solution, we assume directional environment lighting with no inter-reflection. We then solve this bilinear system by iteratively fixing one set of parameters (H or B) and solving a linear least squares problem for the other. Because LTT is sensitive to reflection directions, we also optimize the bump maps along the process.

 

 

DEMO VIDEO


Video: demo (.avi, DiVx 6.2 compressed, ~70MB)

If you can not play the video, please download DivX 6.2 codec

 

Estimation:

 

input                    diffuse map                specular map                   3D shape                  illumination

 

Relighting:

 

changing lighting

changing texture

object insertion

 

 


Updated: May.21, 2006