Take a look at the new version of this page: [ beta version ]. Tell us what you think.

ABSTRACT

Good information design depends on clarifying the meaningful structure in an image. We describe a computational approach to stylizing and abstracting photographs that explicitly responds to this design goal. Our system transforms images into a line-drawing style using bold edges and large regions of constant color. To do this, it represents images as a hierarchical structure of parts and boundaries computed using state-of-the-art computer vision. Our system identifies the meaningful elements of this structure using a model of human perception and a record of a user's eye movements in looking at the photo; the system renders a new image using transformations that preserve and highlight these visual elements. Our method thus represents a new alternative for non-photorealistic rendering both in its visual style, in its approach to visual form, and in its techniques for interaction.

REFERENCES

• 1: Maneesh Agrawala , Chris Stolte, Rendering effective route maps: improving usability through generalization, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.241-249, August 2001 [doi>10.1145/383259.383286]
• 2: Narendra Ahuja, A Transform for Multiscale Image Segmentation by Integrated Edge and Region Detection, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.18 n.12, p.1211-1235, December 1996 [doi>10.1109/34.546258]
• 3: BURT, P., AND ADELSON, E. 1983. The Laplacian pyramid as a compact image code. IEEE Trans. on Communications 31, 4, 532-540.
• 4: CAMPBELL, F., AND ROBSON, J. 1968. Application of Fourier analysis to the visibility of gratings. Journal of Physiology 197, 551-566.
• 5: Christopher M. Christoudias, Synergism in Low Level Vision, Proceedings of the 16 th International Conference on Pattern Recognition (ICPR'02) Volume 4, p.40150, August 11-15, 2002
• 6: Dorin Comaniciu , Peter Meer, Mean Shift: A Robust Approach Toward Feature Space Analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.5, p.603-619, May 2002 [doi>10.1109/34.1000236]
• 7: CURTIS, C. 1999. Non-photorealistic animation. In ACM SIGGRAPH 1999 Course Notes #17 (Section 9).
• 8: Oliver Deussen , Thomas Strothotte, Computer-generated pen-and-ink illustration of trees, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.13-18, July 2000 [doi>10.1145/344779.344792]
• 9: DUCHOWSKI, A., AND VERTEGAAL, R. 2000. Eye-based interaction in graphical systems: Theory and practice. In ACM SIGGRAPH 2000 Course Notes #5.
• 10: DUCHOWSKI, A. 2000. Acuity-matching resolution degradation through wavelet coefficient scaling. IEEE Trans. on Image Processing 9, 8 (Aug.), 1437-1440.
• 11: Frédo Durand , Victor Ostromoukhov , Mathieu Miller , Francois Duranleau , Julie Dorsey, Decoupling Strokes and High-Level Attributes for Interactive Traditional Drawing, Proceedings of the 12th Eurographics Workshop on Rendering Techniques, p.71-82, June 25-27, 2001
• 12: Adam Finkelstein , David H. Salesin, Multiresolution curves, Proceedings of the 21st annual conference on Computer graphics and interactive techniques, p.261-268, July 1994 [doi>10.1145/192161.192223]
• 13: James D. Foley , Andries van Dam , Steven K. Feiner , John F. Hughes, Computer graphics: principles and practice (2nd ed.), Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1990
• 14: Bruce Gooch , Amy Gooch, Non-Photorealistic Rendering, A. K. Peters, Ltd., Natick, MA, 2001
• 15: Amy Gooch , Bruce Gooch , Peter Shirley , Elaine Cohen, A non-photorealistic lighting model for automatic technical illustration, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.447-452, July 1998 [doi>10.1145/280814.280950]
• 16: Paul Haeberli, Paint by numbers: abstract image representations, Proceedings of the 17th annual conference on Computer graphics and interactive techniques, p.207-214, September 1990, Dallas, TX, USA
• 17: HANDFORD, M. 1987. Where's Waldo? Little, Brown and Company.
• 18: HENDERSON, J. M., AND HOLLINGWORTH, A. 1998. Eye movements during scene viewing: An overview. In Eye Guidance in Reading and Scene Perception, G. Underwood, Ed. Elsevier Science Ltd., 269-293.
• 19: IEEE Computer Graphics and Applications Staff, Minimal Graphics, IEEE Computer Graphics and Applications, v.21 n.6, p.18-21, November 2001 [doi>10.1109/38.963456]
• 20: Aaron Hertzmann , Denis Zorin, Illustrating smooth surfaces, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.517-526, July 2000 [doi>10.1145/344779.345074]
• 21: Aaron Hertzmann, Painterly rendering with curved brush strokes of multiple sizes, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.453-460, July 1998 [doi>10.1145/280814.280951]
• 22: Aaron Hertzmann, Paint By Relaxation, Computer Graphics International 2001, p.47-54, July 03-06, 2001
• 23: HOFFMAN, D. D. 1998. Visual intelligence: how we create what we see. Norton.
• 24: JUST, M. A., AND CARPENTER, P. A. 1976. Eye fixations and cognitive processes. Cognitive Psychology 8, 441-480.
• 25: KELLY, D. 1984. Retinal inhomogenity: I. spatiotemporal contrast sensitivity. Journal of the Optical Society of America A 74, 1, 107-113.
• 26: KOENDERINK, J. J., M. A. BOUMAN, A. B. D. M., AND SLAPPENDEL, S. 1978. Perimetry of contrast detection thresholds of moving spatial sine wave patterns. II. the far peripheral visual field (eccentricity 0-50). Journal of the Optical Society of America A 68, 6, 850-854.
• 27: KOENDERINK, J. J. 1984. The structure of images. Biological Cybernetics 50, 363-370.
• 28: KOENDERINK, J. J. 1984. What does the occluding contour tell us about solid shape? Perception 13, 321-330.
• 29: Michael A. Kowalski , Lee Markosian , J. D. Northrup , Lubomir Bourdev , Ronen Barzel , Loring S. Holden , John F. Hughes, Art-based rendering of fur, grass, and trees, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.433-438, July 1999 [doi>10.1145/311535.311607]
• 30: LEYTON, M. 1992. Symmetry, causality, mind. MIT Press.
• 31: Tony Lindeberg, Scale-Space Theory in Computer Vision, Kluwer Academic Publishers, Norwell, MA, 1994
• 32: Peter Litwinowicz, Processing images and video for an impressionist effect, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.407-414, August 1997 [doi>10.1145/258734.258893]
• 33: MACKWORTH, N., AND MORANDI, A. 1967. The gaze selects informative details within pictures. Perception and Psychophysics 2, 547-552.
• 34: MANNOS, J. L., AND SAKRISON, D. J. 1974. The effects of a visual fidelity criterion on the encoding of images. IEEE Trans. on Information Theory 20, 4, 525-536.
• 35: Lee Markosian , Michael A. Kowalski , Daniel Goldstein , Samuel J. Trychin , John F. Hughes , Lubomir D. Bourdev, Real-time nonphotorealistic rendering, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.415-420, August 1997 [doi>10.1145/258734.258894]
• 36: David Marr, Vision: A Computational Investigation into the Human Representation and Processing of Visual Information, Henry Holt and Co., Inc., New York, NY, 1982
• 37: P. Meer , B. Georgescu, Edge Detection with Embedded Confidence, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.23 n.12, p.1351-1365, December 2001 [doi>10.1109/34.977560]
• 38: Sumanta N. Pattanaik , James A. Ferwerda , Mark D. Fairchild , Donald P. Greenberg, A multiscale model of adaptation and spatial vision for realistic image display, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.287-298, July 1998 [doi>10.1145/280814.280922]
• 39: Martin Reddy, Perceptually Optimized 3D Graphics, IEEE Computer Graphics and Applications, v.21 n.5, p.68-75, September 2001
• 40: REGAN, D. 2000. Human Perception of Objects: Early Visual Processing of Spatial Form Defined by Luminance, Color, Texture, Motion and Binocular Disparity. Sinauer.
• 41: ROVAMO, J., AND VIRSU, V. 1979. An estimation and application of the human cortical magnification factor. Experimental Brain Research 37, 495-510.
• 42: Takafumi Saito , Tokiichiro Takahashi, Comprehensible rendering of 3-D shapes, Proceedings of the 17th annual conference on Computer graphics and interactive techniques, p.197-206, September 1990, Dallas, TX, USA
• 43: Anthony Santella , Doug DeCarlo, Abstracted painterly renderings using eye-tracking data, Proceedings of the 2nd international symposium on Non-photorealistic animation and rendering, June 03-05, 2002, Annecy, France [doi>10.1145/508530.508544]
• 44: Michio Shiraishi , Yasushi Yamaguchi, An algorithm for automatic painterly rendering based on local source image approximation, Proceedings of the 1st international symposium on Non-photorealistic animation and rendering, p.53-58, June 05-07, 2000, Annecy, France [doi>10.1145/340916.340923]
• 45: Linda E. Sibert , Robert J. K. Jacob, Evaluation of eye gaze interaction, Proceedings of the SIGCHI conference on Human factors in computing systems, p.281-288, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332445]
• 46: Emanuele Trucco , Alessandro Verri, Introductory Techniques for 3-D Computer Vision, Prentice Hall PTR, Upper Saddle River, NJ, 1998
• 47: Edward Tufte, Envisioning information, Graphics Press, Cheshire, CT, 1990
• 48: Roel Vertegaal, The GAZE groupware system: mediating joint attention in multiparty communication and collaboration, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.294-301, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303065]
• 49: Georges Winkenbach , David H. Salesin, Computer-generated pen-and-ink illustration, Proceedings of the 21st annual conference on Computer graphics and interactive techniques, p.91-100, July 1994 [doi>10.1145/192161.192184]
• 50: YARBUS, A. L. 1967. Eye Movements and Vision. Plenum Press.
• 51: ZEKI, S. 1999. Inner Vision: An Exploration of Art and the Brain. Oxford Univ. Press.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.3 Picture/Image Generation

Additional Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
        I.4.10 Image Representation
            Subjects: Hierarchical

General Terms:
Algorithms, Design

Keywords:
eye-tracking, image simplification, non-photorealistic rendering, visual perception