Reconstruction:

3D Shape, Illumination, Shading, Reflectance, Texture

"Nothing of what is visible, apart from light and color, can be perceived by pure sensation, but only by discernment, inference, and recognition, in addition to sensation." Alhazen, 965-1040 CE

A core problem of vision is the task of inferring the underlying physical world — the shapes and colors of objects, the locations of lights, etc — that gave rise to an observed image. These quantities cannot be directly observed in a single image, which is why a realistic painting can give the impression of a vivid, physical scene, despite being just a collection of paint on a flat canvas.

From a computational perspective, the problem of reconstructing the world from an image is terribly underconstrained. One approach we have explored is to adequately constrain the problem by introducing additional observations, such as multiple images taken under different viewpoints or different illumination conditions. We have also explored the information in a single image due to texture, whose regularities provide insight into the shapes of surfaces. More recently we have begun framing this problem as one of statistical inference, in which we learn models for shape, reflectance, and illumination, and recover the most likely explanation for a single image.

Publications:

	"Intrinsic Scene Properties from a Single RGB-D Image" J. T. Barron & J. Malik.   CVPR 2013    [pdf] [supplementary] [bibtex]
	"On Differential Photometric Reconstruction for Unknown, Isotropic BRDFs" M. Chandraker, J. Bai, & R. Ramamoorthi.   PAMI 2013    [pdf] [bibtex]
	"Compressive Structured Light for Recovering Inhomogeneous Participating Media" J. Gu, S. K. Nayar, E. Grinspun, P. N. Belhumeur, & R. Ramamoorthi.   PAMI 2013    [pdf] [bibtex]
	"Color Constancy, Intrinsic Images, and Shape Estimation" J. T. Barron & J. Malik.   ECCV 2012    [pdf] [supplementary] [bibtex] [movie] [code/data]
	"Shape, Albedo, and Illumination from a Single Image of an Unknown Object" J. T. Barron & J. Malik.   CVPR 2012    [pdf] [supplementary] [bibtex]
	"High-Frequency Shape and Albedo from Shading using Natural Image Statistics" J. T. Barron & J. Malik.   CVPR 2011    [pdf] [bib]
	"A Theory of Differential Photometric Stereo for Unknown Isotropic BRDFs" M. Chandraker, J. Bai, & R. Ramamoorthi.   CVPR 2011    [pdf] [bibtex]
	"Image Quilting for Texture Synthesis and Transfer" A. A. Efros & W. T. Freeman.   SIGGRAPH 2001    [pdf] [more info]
	"Shape from texture and integrability" D. A. Forsyth.   ICCV 2001    [pdf] [ps]
	"Rendering Synthetic Objects into Real Scenes." Y. Yu, A. Ferencz, & J. Malik.   TVCG 2001    [pdf] [more info]
	"Representing and Recognizing the Visual Appearance of Materials using Three-dimensional Textons" T. K. Leung & J. Malik.   IJCV 2000    [pdf] [ps] [more info]
	"Texture Synthesis by Non-Parametric Sampling" A. A. Efros & T. K. Leung.   ICCV 1999    [pdf] [ps] [more info]
	"Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs." Y. Yu, P. E. Debevec, J. Malik, & T. Hawkins. SIGGRAPH 1999    [pdf]
	"Rendering Synthetic Objects into Real Scenes." P. E. Debevec & J. Malik.   SIGGRAPH 1998    [pdf]
	"Recovering Photometric Properties Of Architectural Scenes From Photographs." Y. Yu & J. Malik.   SIGGRAPH 1998    [pdf]
	"On Perpendicular Texture: Why do we see more flowers in the distance?" T. K. Leung and J. Malik.   CVPR 1997    [pdf] [ps]
	"Detecting, Localizing and Grouping Repeated Scene Elements from an Image" T. K. Leung & J. Malik.   ECCV 1996    [pdf] [ps]
	"Computing local surface orientation and shape from texture for curved surfaces." J. Malik & R. Rosenholtz.   IJCV 1997    [pdf] [ps]
	"Recovering High Dynamic Range Radiance Maps from Photographs." P. E. Debevec & J. Malik.   SIGGRAPH 1997    [pdf]
	"Modeling and Rendering Architecture from Photographs." P. E. Debevec, C. J. Taylor, & J. Malik.   SIGGRAPH 1996    [pdf] [video]