A perceptual approach to trimming unstructured lumigraphs

ABSTRACT

We present a novel perceptual method to reduce the visual redundancy of unstructured lumigraphs, an image based representation designed for interactive rendering. We combine features of the unstructured lumigraph algorithm and image fidelity metrics to efficiently rank the perceptual impact of the removal of sub-regions of input views (sub-views). We use a greedy approach to estimate the order in which sub-views should be pruned to minimize perceptual degradation at each step. Renderings using varying numbers of sub-views can then be easily visualized with confidence that the retained sub-views are well chosen, thus facilitating the choice of how many to retain. The regions of the input views that are left are repacked into a texture atlas. Our method takes advantage of any scene geometry information available but only requires a very coarse approximation. We perform a user study to validate its behaviour, as well as investigate the impact of the choice of image fidelity metric. The three metrics considered fall in the physical, statistical and perceptual categories. The overall benefit of our method is the semi-automation of the view selection process, resulting in unstructured lumigraphs that are thriftier in texture memory use and faster to render. (Note to reviewers: a video is available at http://isg.cs.tcd.ie/ymorvan/paper37.avi. The figure occupying the ninth page is intended to appear on a color plate.)

REFERENCES

• 1: Chris Buehler , Michael Bosse , Leonard McMillan , Steven Gortler , Michael Cohen, Unstructured lumigraph rendering, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.425-432, August 2001 [doi>10.1145/383259.383309]
• 2: Coombe, G., Hantak, C., Lastra, A., and Grzeszczuk, R. 2005. Online construction of surface light fields. In Rendering Techniques, 83--90.
• 3: Scott Daly, The visible differences predictor: an algorithm for the assessment of image fidelity, Digital images and human vision, MIT Press, Cambridge, MA, 1993
• 4: Debevec, P., Borshukov, G., and Yu, Y. 1998. Efficient view-dependent image-based rendering with projective texture mapping. In Rendering Techniques 98 Proc. Eurographics Workshop on Rendering.
• 5: Reynald Dumont , Fabio Pellacini , James A. Ferwerda, Perceptually-driven decision theory for interactive realistic rendering, ACM Transactions on Graphics (TOG), v.22 n.2, p.152-181, April 2003 [doi>10.1145/636886.636888]
• 6: Fleishman, S., Cohen-Or, D., and Lischinski, D. 2000. Automatic camera placement for image-based modeling. Computer Graphics Forum 19, 2, 101--110.
• 7: Steven J. Gortler , Radek Grzeszczuk , Richard Szeliski , Michael F. Cohen, The lumigraph, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.43-54, August 1996 [doi>10.1145/237170.237200]
• 8: Hale, J. G. 1998. Texture re-mapping for decimated polygonal meshes. Edinburgh University.
• 9: Benno Heigl , Reinhard Koch , Marc Pollefeys , Joachim Denzler , Luc J. Van Gool, Plenoptic Modeling and Rendering from Image Sequences Taken by Hand-Held Camera, Mustererkennung 1999, 21. DAGM-Symposium, p.94-101, September 15-17, 1999
• 10: Václav Hlavác , Ales Leonardis , Thomás Werner, Automatic Selection of Reference Views for Image-Based Scene Representations, Proceedings of the 4th European Conference on Computer Vision-Volume I, p.526-535, April 15-18, 1996
• 11: Mantiuk, R., Daly, S., Myszkowski, K., and Seidel, H.-P. 2005. Predicting visible differences in high dynamic range images - model and its calibration. In Human Vision and Electronic Imaging X, IS&T/SPIE's 17th Annual Symposium on Electronic Imaging (2005), B. E. Rogowitz, T. N. Pappas, and S. J. Daly, Eds., vol. 5666, 204--214.
• 12: Myszkowski, K., Rokita, P., and Tawara, T. 1999. Perceptually-informed accelerated rendering of high quality walkthrough sequences. In Proceedings of the Tenth Eurographics Workshop on Rendering, 5--18.
• 13: Mahesh Ramasubramanian , Sumanta N. Pattanaik , Donald P. Greenberg, A perceptually based physical error metric for realistic image synthesis, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.73-82, July 1999 [doi>10.1145/311535.311543]
• 14: Schirmacher, H., Heidrich, W., and Seidel, H.-P. 1999. Adaptive acquisition of lumigraphs from synthetic scenes. In Computer Graphics Forum (Eurographics '99), The Eurographics Association and Blackwell Publishers, P. Brunet and R. Scopigno, Eds., vol. 18(3), 151--160.
• 15: William A. Stokes , James A. Ferwerda , Bruce Walter , Donald P. Greenberg, Perceptual illumination components: a new approach to efficient, high quality global illumination rendering, ACM SIGGRAPH 2004 Papers, August 08-12, 2004, Los Angeles, California
• 16: Pere-Pau Vázquez , Miquel Feixas , Mateu Sbert , Wolfgang Heidrich, Viewpoint Selection using Viewpoint Entropy, Proceedings of the Vision Modeling and Visualization Conference 2001, p.273-280, November 21-23, 2001
• 17: Walter, B., Greenberg, D. P., and Pattanaik, S. N. 2002. Using perceptual texture masking for efficient image synthesis. In Eurographics Computer Graphics Forum, vol. 21.
• 18: Wang, Z., Bovik, A. C., and Lu, L. 2002. Why is image quality assessment so difficult? In Proceedings of the IEEE International Conference on Acoustics, Speech, & Signal Processing, vol. 4, 3313--3316.
• 19: Nathaniel Williams , David Luebke , Jonathan D. Cohen , Michael Kelley , Brenden Schubert, Perceptually guided simplification of lit, textured meshes, Proceedings of the 2003 symposium on Interactive 3D graphics, April 27-30, 2003, Monterey, California [doi>10.1145/641480.641503]
• 20: Xin, T., and Gray, R. 2003. Interactive rendering from compressed light fields. IEEE Transactions on Circuits and Systems for Video Technology 13, 11, 1080--1091.
• 21: Yangli Hector Yee , Anna Newman, A perceptual metric for production testing, ACM SIGGRAPH 2004 Sketches, August 08-12, 2004, Los Angeles, California [doi>10.1145/1186223.1186374]
• 22: Hector Yee , Sumanita Pattanaik , Donald P. Greenberg, Spatiotemporal sensitivity and visual attention for efficient rendering of dynamic environments, ACM Transactions on Graphics (TOG), v.20 n.1, p.39-65, Jan. 2001 [doi>10.1145/383745.383748]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.0 General

Keywords:
perceptually-based rendering, realistic rendering