While offline global illumination algorithms like ray tracing or radiosity have long been able to produce very realistic images, these effects are often absent in interactive applications like games. Indeed, it is rare to see realtime images with realistic illumination from widearea light sources such as skylight, realistic materials like velvet, satin, paints, or wood, and shading effects like soft shadows. We seek to bridge the gap between realism and interactivity, allowing for measured illumination and reflectance data , and taking complex light transport effects like soft shadows into account.
Primary Current Participants 
Shadows, Environment and Normal Maps 
Shadow and environment maps are among the oldest techniques to add realism to computergenerated images. However, shadow maps historically have only supported point light sources. We extend them to handle soft shadows from small area sources. Environment maps have historically supported only mirror specular reflections. We use the signalprocessing methods in my PhD thesis to develop efficient algorithms and analytic methods for both Lambertian objects and general materials. Recently, we have shown that similar ideas, and more modern mathematical tools like spherical expectation maximization, can be used to solve the longstanding problem of normal map filtering.
Efficient ImageBased Methods for
Rendering Soft Shadows:
Siggraph 00, pages 375384 We present two efficient imagebased approaches for computation and display of highquality soft shadows from area light sources. Our methods are related to shadow maps and provide the associated benefits. Full Paper: gzipped PS (4M) PDF (1.7M) Talk: PPT (1.8M) 

An Efficient Representation for Irradiance Environment Maps:
Siggraph 01, pages 497500 We use the 9 parameter Lambertian BRDF approximation for analytic rendering of diffuse objects under distant illumination specified by environment maps. Full Paper: gzipped PS (3.4M) PDF (1M) Talk: PPT (1.8M) Video 

Frequency Space Environment Map Rendering:
Siggraph 02, pages 517526 We present a new method for realtime rendering of objects with complex isotropic BRDFs under distant natural illumination, as specified by an environment map. Our approach is based on our signalprocessing framework. Full Paper: gzipped PS (4.2M) PDF (3.3M) 


Frequency Domain Normal Map Filtering
SIGGRAPH 07. Filtering is critical for representing imagebased detail, such as textures or normal maps, across a variety of scales. While mipmapping textures is commonplace, accurate normal map filtering remains a challenging problem because of nonlinearities in shadingwe cannot simply average nearby surface normals. In this paper, we show analytically that normal map filtering can be formalized as a spherical convolution of the normal distribution function (NDF) and the BRDF, for a large class of common BRDFs such as Lambertian, microfacet and factored measurements. Our practical algorithms leverage a significant body of previous work that has studied lightingBRDF convolution. We show how spherical harmonics can be used to filter the NDF for Lambertian and lowfrequency specular BRDFs, while spherical von MisesFisher distributions can be used for highfrequency materials. Paper: PDF Video (103M) 
AllFrequency Precomputed Light Transport 
More generally, we would be able to simultaneously get the effects of natural lighting and soft shadows at all frequencies. We would like to be able to interactively change the lighting or viewpoint for applications like lighting design or games. Our approach is to precompute information of interest, such as the appearance of a scene under all possible lighting conditions. The challenge is then how to compactly and efficiently represent this information for interactive rendering. We do so by developing new signalprocessing methods using wavelets, which are ideally suited to handle all frequencies, from broad area sources to sharp specular materials. Most recently, we have used similar techniques as for precomputationbased relighting for material design and BRDF editing.
AllFrequency Shadows Using NonLinear Wavelet Lighting Approximation
Siggraph 03, pages 376381 We present a method, based on precomputed light transport, for realtime rendering of objects under allfrequency, timevarying illumination represented as a highresolution environment map. For accurate rendering, using nonlinear wavelets is an order of magnitude faster than using linear spherical harmonics, the current best technique. PDF (1M) Video (42MB) 

Triple Product Wavelet Integrals for AllFrequency Relighting
Siggraph 04, pages 475485 We propose a new mathematical and computational analysis of precomputed light transport. We use factored forms, separately precomputing the effects of visibility and material properties. Rendering then requires computing triple product integrals at each vertex, involving the lighting, visibility and BRDF. Our main contribution is a general analysis of these triple products likely to have broad applicability in computer graphics and numerical analysis. PDF (5M) 

RealTime BRDF Editing in Complex Lighting
Siggraph 06, pages 945954 We develop the theory and algorithms to for the first time allow users to edit BRDFs in real time to design materials in their final placement in a scene with complex natural illumination and cast shadows. PDF (20M) Video (59M) 


Exploiting Temporal Coherence for Incremental AllFrequency Relighting
EGSR 06, pages 151160.
Current PRT methods exploit spatial coherence of the lighting (such as with wavelets) and of light transport (such as with CPCA). We consider a significant, yet unexplored form of coherence, temporal coherence of the lighting from frame to frame. We achieve speedups of 3x4x over conventional PRT with minimal implementation effort, and can trivially be added to almost any existing PRT algorithm. Paper: PDF 

4D Compression and Relighting with HighResolution Light Transport Matrices
ACM Symposium on Interactive 3D graphics 2007, pages 8188. We use a 4D wavelet transform for relighting with allfrequency illumination. A key observation is that a standard 4D wavelet transform can actually inflate portions of the light transport matrix. Therefore, we present an adaptive 4D wavelet transform that terminates at a level that avoids inflation and maximizes sparsity in the matrix data. Finally, we present an algorithm for fast relighting from adaptively compressed transport matrices. Paper: PDF Video 

A Precomputed Polynomial Representation for Interactive BRDF Editing with
Global Illumination
ACM Transactions on Graphics 27(2), Article 13, pages 113.
Presented at SIGGRAPH 2008.
We develop a mathematical framework and practical algorithms to edit BRDFs with global illumination in a complex scene. A key challenge is that light transport for multiple bounces is nonlinear in the scene BRDFs, while current PRT frameworks assume linear light transport. We address this by developing a new bilinear representation of the reflection operator, deriving a polynomial multibounce tensor precomputed framework, and reducing the complexity of further bounces. Paper: PDF Video (7M) 

An Analysis of the InOut BRDF Factorization
for ViewDependent Relighting
EuroGraphics Symposium on Rendering, 2008. We analyze the popular inout BRDF factorization for viewdependent relighting theoretically. We show that for the Phong BRDFs, the factors or eigenmodes are simply spherical harmonic basis functions, and an exact frequency space analysis is possible. The number of terms needed grows linearly with the Phong exponent or quadratically with the frequency cutoff for reflected or halfangle dependence. The theory allows for setting the number of BRDF terms, and also indicates that high frequency specularity is still out of reach of current algorithms. Paper: PDF Video (18M) 
Affine Double and Triple Product Wavelet Integrals for Rendering
ACM Transactions on Graphics 28(2), Article 14, pages 117, Apr 2009. Many problems in computer graphics involve integrations of products of functions. Double and tripleproduct integrals are commonly used in applications such as allfrequency relighting or importance sampling, but are limited to distant illumination. In contrast, nearfield lighting from planar area lights involves an affine transform of the source radiance at different points in space. Our main contribution is a novel affine double and tripleproduct integral theory. Paper: PDF Video (AVI 16M) 

PrecomputationBased Rendering
Foundations and Trends in Computer Graphics and Vision 3(4), 281369. Precomputationbased relighting and radiance transfer has a long history with a spurt of renewed interest, including adoption in commercial video games, due to recent mathematical developments and hardware advances. In this survey, we describe the mathematical foundations, history, current research and future directions for precomputationbased rendering. Paper: PDF 