Environment matting extensions: towards higher accuracy and real-time capture

ABSTRACT

Environment matting is a generalization of traditional bluescreen matting. By photographing an object in front of a sequence of structured light backdrops, a set of approximate light-transport paths through the object can be computed. The original environment matting research chose a middle ground—using a moderate number of photographs to produce results that were reasonably accurate for many objects. In this work, we extend the technique in two opposite directions: recovering a more accurate model at the expense of using additional structured light backdrops, and obtaining a simplified matte using just a single backdrop. The first extension allows for the capture of complex and subtle interactions of light with objects, while the second allows for video capture of colorless objects in motion.

REFERENCES

• 1: P. J. Besl, Active optical range imaging sensors, Advances in Machine Vision, Springer-Verlag New York, Inc., New York, NY, 1988
• 2: James F. Blinn , Martin E. Newell, Texture and reflection in computer generated images, Communications of the ACM, v.19 n.10, p.542-547, Oct. 1976 [doi>10.1145/360349.360353]
• 3: G. Chazan and N. Kiryati. Pyramidal intensity ratio depth sensor. Technical Report 121, Center for Communication and Information Technologies, Department of Electrical Engineering, Technion, Haifa, Israel, October 1995.
• 4: B. Curless , M. Levoy, Better optical triangulation through spacetime analysis, Proceedings of the Fifth International Conference on Computer Vision, p.987, June 20-23, 1995
• 5: Paul E. Debevec , Jitendra Malik, Recovering high dynamic range radiance maps from photographs, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.369-378, August 1997 [doi>10.1145/258734.258884]
• 6: Gerd Hfiusler and Dieter Ritter. Parallel three-dimensional sensing by colorcoded triangulation. Applied Optics, 32(35):7164-7169, December 1993.
• 7: David J. Heeger , James R. Bergen, Pyramid-based texture analysis/synthesis, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.229-238, September 1995 [doi>10.1145/218380.218446]
• 8: E. Horn , N. Kiryati, Toward optimal structured light patterns, Proceedings of the International Conference on Recent Advances in 3-D Digital Imaging and Modeling, p.28, May 12-15, 1997
• 9: T. Kanade, A. Gruss, and L. Carley. A very fast VLSI rangefinder. In 1991 IEEE International Conference on Robotics and Automation, volume 39, pages 1322-1329, April 1991.
• 10: P. Perona , J. Malik, Scale-Space and Edge Detection Using Anisotropic Diffusion, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.12 n.7, p.629-639, July 1990 [doi>10.1109/34.56205]
• 11: Thomas Porter , Tom Duff, Compositing digital images, Proceedings of the 11th annual conference on Computer graphics and interactive techniques, p.253-259, January 1984
• 12: William H. Press , Saul A. Teukolsky , William T. Vetterling , Brian P. Flannery, Numerical recipes in C (2nd ed.): the art of scientific computing, Cambridge University Press, New York, NY, 1992
• 13: K. Sato and S. Inokuchi. Three-dimensional surface measurement by space encoding range imaging. Journal of Robotic Systems, 2:27-39, 1985.
• 14: E. Schubert, Fast 3D Object Recognition Using Multiple Color Coded Illumination, Proceedings of the 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '97) -Volume 4, p.3057, April 21-24, 1997
• 15: Alvy Ray Smith , James F. Blinn, Blue screen matting, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.259-268, August 1996 [doi>10.1145/237170.237263]
• 16: Gregory J. Ward, Measuring and modeling anisotropic reflection, ACM SIGGRAPH Computer Graphics, v.26 n.2, p.265-272, July 1992
• 17: George Wolberg, Digital Image Warping, IEEE Computer Society Press, Los Alamitos, CA, 1994
• 18: Yung Yu Chuang, Douglas E. Zongker, Joel Hindorff, Brian Curless, David H. Salesin, and Richard Szeliski. Environment matting extensions: Towards higher accuracy and real-time capture. Technical Report 2000-05-01, University of Washington, 2000.
• 19: Douglas E. Zongker , Dawn M. Werner , Brian Curless , David H. Salesin, Environment matting and compositing, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.205-214, July 1999 [doi>10.1145/311535.311558]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.10 Vision and Scene Understanding
            Subjects: Modeling and recovery of physical attributes

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.8 Problem Solving, Control Methods, and Search
          Subjects: Heuristic methods
  I.3 COMPUTER GRAPHICS
      I.3.3 Picture/Image Generation
          Subjects: Display algorithms
      I.3.7 Three-Dimensional Graphics and Realism
          Subjects: Color, shading, shadowing, and texture

General Terms:
Algorithms, Design, Experimentation, Measurement, Performance, Reliability, Theory

Keywords:
alpha channel, augmented reality, blue spill, blue-screen matting, clip art, colored transparency, environment map, environment matte, image-based rendering, real-time capture, reflection, refraction