IRW: an incremental representation for image-based walkthroughs

ABSTRACT

We present a new representation for image-based interactive walk-throughs. The target applications reconstruct a scene from novel viewpoints using samples from a spatial image dataset collected from a plane at eye-level. These datasets consist of pose augmented 2D images and often have a very large number of samples. Our representation exploits spatial coherence and rearranges the input samples as epipolar images. The base unit corresponds to a column of the original image that can be individually addressed and accessed. The overall representation, IRW, supports incremental updates, efficient encoding, scalable performance, and selective inclusion used by different reconstruction algorithms. We demonstrate the performance of our representation on a synthetic as well as a real-world environment.

REFERENCES

• 1: E. Adelson and J. Bergen. The plenoptic function and the elements of early vision. In Computational Models of Visual Processing, pages 3--20. MIT Press, 1991.
• 2: Daniel Aliaga , Jon Cohen , Andrew Wilson , Eric Baker , Hansong Zhang , Carl Erikson , Kenny Hoff , Tom Hudson , Wolfgang Stuerzlinger , Rui Bastos , Mary Whitton , Fred Brooks , Dinesh Manocha, MMR: an interactive massive model rendering system using geometric and image-based acceleration, Proceedings of the 1999 symposium on Interactive 3D graphics, p.199-206, April 26-29, 1999, Atlanta, Georgia, United States [doi>10.1145/300523.300554]
• 3: Daniel G. Aliaga , Anselmo Lastra, Automatic image placement to provide a guaranteed frame rate, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.307-316, July 1999 [doi>10.1145/311535.311574]
• 4: Daniel G. Aliaga , Ingrid Carlbom, Plenoptic stitching: a scalable method for reconstructing 3D interactive walk throughs, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.443-450, August 2001 [doi>10.1145/383259.383311]
• 5: R. C. Bolles, H. H. Baker, and D. H. Marimont. Epipolar-plane image analysis: An approach to determining structure from motion. International Journal of Computer Vision, 1:7--55, 1987.
• 6: Shenchang Eric Chen , Lance Williams, View interpolation for image synthesis, Proceedings of the 20th annual conference on Computer graphics and interactive techniques, p.279-288, September 1993 [doi>10.1145/166117.166153]
• 7: Shenchang Eric Chen, QuickTime VR: an image-based approach to virtual environment navigation, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.29-38, September 1995 [doi>10.1145/218380.218395]
• 8: L. Darsa, B. Costa, and A. Varshney. Walkthroughs of complex environments using image-based simplification. Computer and Graphics, 22(1):55--69, 1998.
• 9: X. Decoret, G. Schaufler, F. Sillion, and J. Dorsey. Multi-layered impostors for accelerated rendering. Computer Graphics Forum, 18(3), 1999.
• 10: Steven J. Gortler , Radek Grzeszczuk , Richard Szeliski , Michael F. Cohen, The lumigraph, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.43-54, August 1996 [doi>10.1145/237170.237200]
• 11: J. Grossman and W. J. Dally. Point sample rendering. Eurographics Workshop on Rendering, pages 181--192, 1998.
• 12: ISO/IEC. ISO/IEC International Standard 11172, August 1993.
• 13: ISO/IEC. ISO/IEC International Standard 13818, July 1995.
• 14: ISO/IEC. ISO/IEC International Standard 15444, Final Committee Draft, March 2000.
• 15: Sing Bing Kang , Richard Szeliski, 3-D Scene Data Recovery using Omnidirectional Multibaseline Stereo, Proceedings of the 1996 Conference on Computer Vision and Pattern Recognition (CVPR '96), p.364, June 18-20, 1996
• 16: Paul Lalonde , Alain Fournier, Interactive rendering of wavelet projected light fields, Proceedings of the 1999 conference on Graphics interface '99, p.107-114, September 1999, Kingston, Ontario, Canada
• 17: Marc Levoy, Polygon-assisted JPEG and MPEG compression of synthetic images, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.21-28, September 1995 [doi>10.1145/218380.218392]
• 18: Marc Levoy , Pat Hanrahan, Light field rendering, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.31-42, August 1996 [doi>10.1145/237170.237199]
• 19: Paulo W. C. Maciel , Peter Shirley, Visual navigation of large environments using textured clusters, Proceedings of the 1995 symposium on Interactive 3D graphics, p.95-ff., April 09-12, 1995, Monterey, California, United States [doi>10.1145/199404.199420]
• 20: M. Magnor and B. Girod. Model-based coding of multi-viewpoint imagery. SPIE Conference on Visual Communications and Image Processing, pages 14--22, 2000.
• 21: N. Max and K. Ohsaki. Rendering trees from precomputed Z-buffer views. In Eurographics Rendering Workshop 1995, 1995.
• 22: Leonard McMillan , Gary Bishop, Plenoptic modeling: an image-based rendering system, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.39-46, September 1995 [doi>10.1145/218380.218398]
• 23: G. Miller, S. Rubin, and D. Poncelen. Lazy decompression of surface light fields for pre-computer global illumination. Proc. of Eurographics Workshop on Rendering, pages 281--292, 1998.
• 24: William B. Pennebaker , Joan L. Mitchell, JPEG Still Image Data Compression Standard, Kluwer Academic Publishers, Norwell, MA, 1992
• 25: Hanspeter Pfister , Matthias Zwicker , Jeroen van Baar , Markus Gross, Surfels: surface elements as rendering primitives, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.335-342, July 2000 [doi>10.1145/344779.344936]
• 26: K. R. Rao , P. Yip, Discrete cosine transform: algorithms, advantages, applications, Academic Press Professional, Inc., San Diego, CA, 1990
• 27: Jonathan Shade , Steven Gortler , Li-wei He , Richard Szeliski, Layered depth images, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.231-242, July 1998 [doi>10.1145/280814.280882]
• 28: Heung-Yeung Shum , Li-Wei He, Rendering with concentric mosaics, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.299-306, July 1999 [doi>10.1145/311535.311573]
• 29: F. Sillion, G. Drettakis, and B. Bodelet. Efficient impostor manipulation for real-time visualization of urban scenery. In Computer Graphics Forum, volume 16, 1997.
• 30: Richard Szeliski , Heung-Yeung Shum, Creating full view panoramic image mosaics and environment maps, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.251-258, August 1997 [doi>10.1145/258734.258861]
• 31: T. Takahashi, H. Kawasaki, K. Ikeuchi, and M. Sakauchi. Arbitrary view position and direction rendering for large-scale scenes. IEEE CVPR, pages 296--303, 2000.
• 32: Camillo J. Taylor, Video Plus, Proceedings of the IEEE Workshop on Omnidirectional Vision, p.3, June 16-16, 2000
• 33: I. T. Union. ITU-T H. 261: Video codec for audiovisual services at p x 64 kbits., March 1993.
• 34: Martin Vetterli , Jelena Kovačevic, Wavelets and subband coding, Prentice-Hall, Inc., Upper Saddle River, NJ, 1995
• 35: A. Wilson. Spatial Representations of Image-Based Impostors for Interactive Walkthroughs. PhD thesis, Department of Computer Science, University of North Carolina at Chapel Hill, 2002.
• 36: Andrew Wilson , Ming C. Lin , Boon-Lock Yeo , Minerva Yeung , Dinesh Manocha, A video-based rendering acceleration algorithm for interactive walkthroughs, Proceedings of the eighth ACM international conference on Multimedia, p.75-83, October 2000, Marina del Rey, California, United States [doi>10.1145/354384.354431]
• 37: Andrew Wilson , Ketan Mayer-Patel , Dinesh Manocha, Spatially-encoded far-field representations for interactive walkthroughs, Proceedings of the ninth ACM international conference on Multimedia, September 30-October 05, 2001, Ottawa, Canada [doi>10.1145/500141.500193]
• 38: Yunnan Wu , Cha Zhang , Jin Li , Jizheng Xu, Smart rebinning for compression of concentric mosaics, Proceedings of the eighth ACM international conference on Multimedia, p.201-209, October 2000, Marina del Rey, California, United States [doi>10.1145/354384.354476]