The Convergence of Graphics and Vision

ABSTRACT

Computer graphics and computer vision are inverse problems. Traditional computer graphics starts with input geometric models and produces image sequences. Traditional computer vision starts with input image sequences and produces geometric models. Lately, there has been a meeting in the middle, and the center--the prize--is to create stunning, photorealistic images in real time. Vision researchers now work from images backward just as far as necessary to create models that capture a scene without going to full geometric models. Graphics researchers now work with hybrid geometry and image models. These models use images as partial results, reusing them to take advantage of similarities in the image stream. There are several current trends that make this an exciting time for image synthesis: The combined graphics and vision approaches have a hybrid vigor, much of which stems from sampled representations. This use of captured scenes (enhanced by vision research) yields richer rendering and modeling methods (for graphics) than methods that synthesize everything from scratch. Exploiting temporal and spatial coherence (similarities in images) via the use of layers and other techniques is boosting runtime performance. The explosion in PC graphics performance is making powerful computational techniques more practical. This article surveys cutting edge work in this exciting field, some of which will debut at Siggraph 98.

REFERENCES

• 1: S. Kang, <i>A Survey of Image-Based Rendering Techniques,</i> Tech. Report CRL 97/4, Digital Equipment Corp., Cambridge Research Lab, Cambridge, Mass., 1997.
• 2: Zhengyou Zhang, Image-Based Geometrically-Correct Photorealistic Scene/Object Modeling (IBPhM): A Review, Proceedings of the Third Asian Conference on Computer Vision-Volume II, p.340-349, January 08-10, 1998
• 3: Shenchang Eric Chen , Lance Williams, View interpolation for image synthesis, Proceedings of the 20th annual conference on Computer graphics and interactive techniques, p.279-288, September 1993 [doi>10.1145/166117.166153]
• 4: R. Szeliski, "Video Mosaics for Virtual Environments," <i>IEEE Computer Graphics and Applications,</i> Mar. 1996, pp. 22-30.
• 5: Leonard McMillan , Gary Bishop, Plenoptic modeling: an image-based rendering system, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.39-46, September 1995 [doi>10.1145/218380.218398]
• 6: Steven M. Seitz , Charles R. Dyer, View morphing, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.21-30, August 1996 [doi>10.1145/237170.237196]
• 7: Jonathan Shade , Steven Gortler , Li-wei He , Richard Szeliski, Layered depth images, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.231-242, July 1998 [doi>10.1145/280814.280882]
• 8: Paul Rademacher , Gary Bishop, Multiple-center-of-projection images, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.199-206, July 1998 [doi>10.1145/280814.280871]
• 9: Steven Molnar , John Eyles , John Poulton, PixelFlow: high-speed rendering using image composition, Proceedings of the 19th annual conference on Computer graphics and interactive techniques, p.231-240, July 1992
• 10: Matthew Regan , Ronald Pose, Priority rendering with a virtual reality address recalculation pipeline, Proceedings of the 21st annual conference on Computer graphics and interactive techniques, p.155-162, July 1994 [doi>10.1145/192161.192192]
• 11: Paulo W. C. Maciel , Peter Shirley, Visual navigation of large environments using textured clusters, Proceedings of the 1995 symposium on Interactive 3D graphics, p.95-ff., April 09-12, 1995, Monterey, California, United States [doi>10.1145/199404.199420]
• 12: Gernot Schaufler, Nailboards: A Rendering Primitive for Image Caching in Dynamic Scenes, Proceedings of the Eurographics Workshop on Rendering Techniques '97, p.151-162, June 16-18, 1997
• 13: John Snyder , Jed Lengyel, Visibility sorting and compositing without splitting for image layer decompositions, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.219-230, July 1998 [doi>10.1145/280814.280878]
• 14: Paul Debevec, Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.189-198, July 1998 [doi>10.1145/280814.280864]
• 15: Yizhou Yu , Jitendra Malik, Recovering photometric properties of architectural scenes from photographs, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.207-217, July 1998 [doi>10.1145/280814.280874]
• 16: Homan Igehy , Gordon Stoll , Pat Hanrahan, The design of a parallel graphics interface, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.141-150, July 1998 [doi>10.1145/280814.280837]
• 17: Frederic Pighin, Modeling and animation of realistic facial expressions, ACM SIGGRAPH 98 Electronic art and animation catalog, p.149, July 19-24, 1998, Orlando, Florida, United States [doi>10.1145/281388.281935]
• 18: Brian Guenter , Cindy Grimm , Daniel Wood , Henrique Malvar , Fredric Pighin, Making faces, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.55-66, July 1998 [doi>10.1145/280814.280822]
• 19: Tony DeRose , Michael Kass , Tien Truong, Subdivision surfaces in character animation, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.85-94, July 1998 [doi>10.1145/280814.280826]
• 20: Douglas DeCarlo , Dimitris Metaxas , Matthew Stone, An anthropometric face model using variational techniques, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.67-74, July 1998 [doi>10.1145/280814.280823]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.0 General

Additional Classification:
  A. General Literature
  A.1 INTRODUCTORY AND SURVEY

  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.0 General

General Terms:
Design, Performance