3DDI Abstract

3DDI Project
Abstract:

The purpose of this project is to develop the science base and a spectrum of tools for fast, realistic and direct interaction between humans and 3d content. The effort is heavily multi-disciplinary spanning optical materials, solid-state devices, computational architecture, software, algorithms, mathematical modeling and human-computer interaction. We believe that close collaboration between the exceptional scientists involved in this proposal will bring the breakthroughs needed for the leap from 2d to comprehensive 3d interface technology, which we call 3d direct manipulation. The cooperation leading to this proposal has already led to some exciting and novel devices (e.g. the depth camera described in the acquisition section and used in the scenarios). We have conceptualized the project as a pipeline from acquisition to display, including (i) acquisition (ii) modelling (iii) rendering (iv) display and (v) applications.

Acquisition research includes a novel real-time depth camera design, made possible by recent research in large arrays of tunable lasers. This scanner should ultimately provide a depth image at typical image resolutions (500 by 500) in real-time. For large environments (building exteriors) in daylight where the scanner is not feasible, ordinary photographs from multiple views provide input data for 3d reconstruction. Previous work by one PI on constructing texture-mapped 3d models from multiple views will be extended to depth-camera data.

Modeling includes radiance modeling (shading, shadows, mutual illumination) and motion modeling. Previous work by two PIs on radiance modeling will be extended to efficient but accurate global illumination models. Our physical simulation method with full-contact physics generates the most physically realistic motion among current simulation methods.

Three dimensional rendering for holographic or lenticular displays requires frame-rate generation of many views. Rendering research will include further enhancements to the ``optic flow'' hardware renderer invented by one of the PIs, and special hardware to support the volumetric display.

We will study 3d representation using three display types: (i) an improved lenticular (specular-optical) display (ii) a holographic display using acoustic-optical modulators (iii) a volumetric scattering display using cholesteric liquid crystals. Of these, two prototypes have already been constructed, and one is to be developed in this project. The lenticular display and the holographic display are prototypes at the Spatial Imaging lab at MIT. The third is a volumetric self-occluding scattering display to be developed at UC Berkeley.

We will research several applications of 3d technology, namely tele-medecine, collaboration, and crisis simulation.

Last updated, Wednesday, 08-Jul-1998 15:28:48 PDT.
Mail questions or comments to jfc@cs.berkeley.edu.