Image-based visual hulls

ABSTRACT

In this paper, we describe an efficient image-based approach to computing and shading visual hulls from silhouette image data. Our algorithm takes advantage of epipolar geometry and incremental computation to achieve a constant rendering cost per rendered pixel. It does not suffer from the computation complexity, limited resolution, or quantization artifacts of previous volumetric approaches. We demonstrate the use of this algorithm in a real-time virtualized reality application running off a small number of video streams.

REFERENCES

• 1: M. Bichsel, Segmenting Simply Connected Moving Objects in a Static Scene, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.16 n.11, p.1138-1142, November 1994 [doi>10.1109/34.334396]
• 2: Edmond Boyer , Marie-Odile Berger, 3D Surface Reconstruction Using Occluding Contours, International Journal of Computer Vision, v.22 n.3, p.219-233, March/April 1997 [doi>10.1023/A:1007978616082]
• 3: Shenchang Eric Chen , Lance Williams, View interpolation for image synthesis, Proceedings of the 20th annual conference on Computer graphics and interactive techniques, p.279-288, September 1993 [doi>10.1145/166117.166153]
• 4: Shenchang Eric Chen, QuickTime VR: an image-based approach to virtual environment navigation, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.29-38, September 1995 [doi>10.1145/218380.218395]
• 5: Brian Curless , Marc Levoy, A volumetric method for building complex models from range images, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.303-312, August 1996 [doi>10.1145/237170.237269]
• 6: Paul E. Debevec , Camillo J. Taylor , Jitendra Malik, Modeling and rendering architecture from photographs: a hybrid geometry- and image-based approach, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.11-20, August 1996 [doi>10.1145/237170.237191]
• 7: Debevec, P.E., Y. Yu, and G. D. Borshukov, "Efficient View- Dependent Image-based Rendering with Projective Texture Mapping." Proc. of EGRW 1998 (June 1998).
• 8: Paul Ernest Debevec , Jitendra Malik, Modeling and rendering architecture from photographs, 1996
• 9: Olivier Faugeras, Three-dimensional computer vision: a geometric viewpoint, MIT Press, Cambridge, MA, 1993
• 10: Friedman, N. and S. Russel. "Image Segmentation in Video Sequences." Proc 13th Conference on Uncertainty in Artifical Intelligence (1997).
• 11: Jack Goldfeather , Jeff P M Hultquist , Henry Fuchs, Fast constructive-solid geometry display in the pixel-powers graphics system, Proceedings of the 13th annual conference on Computer graphics and interactive techniques, p.107-116, August 1986
• 12: Steven J. Gortler , Radek Grzeszczuk , Richard Szeliski , Michael F. Cohen, The lumigraph, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.43-54, August 1996 [doi>10.1145/237170.237200]
• 13: Takeo Kanade , Peter Rander , P. J. Narayanan, Virtualized Reality: Constructing Virtual Worlds from Real Scenes, IEEE MultiMedia, v.4 n.1, p.34-47, January 1997 [doi>10.1109/93.580394]
• 14: A. Laurentini, The Visual Hull Concept for Silhouette-Based Image Understanding, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.16 n.2, p.150-162, February 1994 [doi>10.1109/34.273735]
• 15: Marc Levoy , Pat Hanrahan, Light field rendering, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.31-42, August 1996 [doi>10.1145/237170.237199]
• 16: William E. Lorensen , Harvey E. Cline, Marching cubes: A high resolution 3D surface construction algorithm, Proceedings of the 14th annual conference on Computer graphics and interactive techniques, p.163-169, August 1987
• 17: Leonard McMillan , Gary Bishop, Plenoptic modeling: an image-based rendering system, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.39-46, September 1995 [doi>10.1145/218380.218398]
• 18: Leonard McMillan, Jr., An image-based approach to three-dimensional computer graphics, University of North Carolina at Chapel Hill, Chapel Hill, NC, 1997
• 19: Saied Moezzi , Arun Katkere , Don Y. Kuramura , Ramesh Jain, Reality Modeling and Visualization from Multiple Video Sequences, IEEE Computer Graphics and Applications, v.16 n.6, p.58-63, November 1996 [doi>10.1109/38.544073]
• 20: P. J. Narayanan , Peter W. Rander , Takeo Kanade, Constructing Virtual Worlds Using Dense Stereo, Proceedings of the Sixth International Conference on Computer Vision, p.3, January 04-07, 1998
• 21: Stephen Pollard , Sean Hayes, View synthesis by edge transfer with application to the generation of immersive video objects, Proceedings of the ACM symposium on Virtual reality software and technology, p.91-98, November 02-05, 1998, Taipei, Taiwan [doi>10.1145/293701.293713]
• 22: Michael Potmesil, Generating octree models of 3D objects from their silhouettes in a sequence of images, Computer Vision, Graphics, and Image Processing, v.40 n.1, p.1-29, Oct. 1987 [doi>10.1016/0734-189X(87)90053-3]
• 23: Peter Rander , P. J. Narayanan , Takeo Kanade, Virtualized reality: constructing time-varying virtual worlds from real world events, Proceedings of the 8th conference on Visualization '97, p.277-ff., October 18-24, 1997, Phoenix, Arizona, United States
• 24: Ari Rappoport , Steven Spitz, Interactive Boolean operations for conceptual design of 3-D solids, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.269-278, August 1997 [doi>10.1145/258734.258865]
• 25: Roth, S. D. "Ray Casting for Modeling Solids." Computer Graphics and Image Processing, 18 (February 1982), 109-144.
• 26: Saito, H. and T. Kanade. "Shape Reconstruction in Projective Grid Space from a Large Number of Images." Proc. of CVPR, (1999).
• 27: Steven M. Seitz , Charles R. Dyer, Photorealistic Scene Reconstruction by Voxel Coloring, Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97), p.1067, June 17-19, 1997
• 28: Seuss, D. "The Cat in the Hat," CBS Television Special (1971).
• 29: Richard Szeliski, Rapid octree construction from image sequences, CVGIP: Image Understanding, v.58 n.1, p.23-32, July 1993 [doi>10.1006/ciun.1993.1029]
• 30: Vedula, S., P. Rander, H. Saito, and T. Kanade. "Modeling, Combining, and Rendering Dynamic Real-World Events from Image Sequences." Proc. 4~h Intl. Conf. on Virtual Systems and Multimedia (Nov 1998).

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.5 Computational Geometry and Object Modeling
          Subjects: Constructive solid geometry (CSG)**

Additional Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.3 Picture/Image Generation
          Subjects: Display algorithms

General Terms:
Algorithms, Design, Measurement, Performance, Theory

Keywords:
computer vision, constructive solid geometry, image-based rendering, misc. rendering algorithms