Take a look at the new version of this page: [ beta version ]. Tell us what you think.

ABSTRACT

We present an interactive method for cropping photographs given minimal information about important content location, provided by eye tracking. Cropping is formulated in a general optimization framework that facilitates adding new composition rules, and adapting the system to particular applications. Our system uses fixation data</ to identify important image content and compute the best crop for any given aspect ratio or size, enabling applications such as automatic snapshot recomposition, adaptive documents, and thumbnailing. We validate our approach with studies in which users compare our crops to ones produced by hand and by a completely automatic approach. Experiments show that viewers prefer our gaze-based crops to uncropped images and fully automatic crops.

REFERENCES

• 1: Arnheim, R. Art and Visual Perception. University of California Press, 1974.
• 2: Arnheim, R. The Power of the Center. University of California Press, 1988.
• 3: Banerjee, S. Composition Guided Image Aquisition. PhD thesis, University of Texas at Austin, 2004.
• 4: Byers, Z., Dixon, M., Smart, W. D., and Grimm, C. M. Say cheese!: Experiences with a robot photographer. Proceedings of the Fifteenth Innovative Applications of Artificial Intelligence Conference (IAAI-03), Acapulco, Mexico.
• 5: Chen, L., Xie, X., Fan, X., Ma, W., Shang, H., and Zhou, H. A visual attention mode for adapting images on small displays. MSR-TR-2002-125 Microsoft Research, Redmond, WA (2002).
• 6: Christopher M. Christoudias, Synergism in Low Level Vision, Proceedings of the 16 th International Conference on Pattern Recognition (ICPR'02) Volume 4, p.40150, August 11-15, 2002
• 7: Franklin C. Crow, Summed-area tables for texture mapping, Proceedings of the 11th annual conference on Computer graphics and interactive techniques, p.207-212, January 1984
• 8: David, H. A. The method of paired comparisons. Charles Griffin and Company, London, 1969.
• 9: Doug DeCarlo , Anthony Santella, Stylization and abstraction of photographs, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 10: Duchowski, A. Acuity-matching resolution degradation through wavelet coefficient scaling. IEEE Trans. on Image Processing 9, 8 (2000), 1437--1440.
• 11: James D. Foley , Andries van Dam , Steven K. Feiner , John F. Hughes, Computer graphics: principles and practice (2nd ed.), Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1990
• 12: Bruce Gooch , Erik Reinhard , Chris Moulding , Peter Shirley, Artistic Composition for Image Creation, Proceedings of the 12th Eurographics Workshop on Rendering Techniques, p.83-88, June 25-27, 2001
• 13: Graham, D. Composing Pictures. Van Nostrand Reinhold, 1970.
• 14: Grill, T., and Scanlon, M. Photographic Composition Guidelines for Total image control through effective design. AMPHOTO, 1988.
• 15: Itti, L., and Koch, C. A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Research 40 (2000), 1489--1506.
• 16: Laurent Itti , Christof Koch , Ernst Niebur, A Model of Saliency-Based Visual Attention for Rapid Scene Analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.20 n.11, p.1254-1259, November 1998 [doi>10.1109/34.730558]
• 17: Jacob, R. J. Eye-movement-based human-computer interaction techniques: Toward non-command interfaces. 151--190.
• 18: Charles Jacobs , Wilmot Li , Evan Schrier , David Bargeron , David Salesin, Adaptive grid-based document layout, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003
• 19: Kendall, M. G. On the method of paired comparisons. Biometrika 31 (1940), 324 --345.
• 20: Stan Z. Li , Long Zhu , ZhenQiu Zhang , Andrew Blake , HongJiang Zhang , Harry Shum, Statistical Learning of Multi-view Face Detection, Proceedings of the 7th European Conference on Computer Vision-Part IV, p.67-81, May 28-31, 2002
• 21: Yin Li , Jian Sun , Chi-Keung Tang , Heung-Yeung Shum, Lazy snapping, ACM Transactions on Graphics (TOG), v.23 n.3, August 2004
• 22: Locher, P. J. The contribution of eye-movement research to an understanding of the nature of pictorial balance perception: a review of the literature. Empirical Studies of the Arts 14, 2 (1996), 146--163.
• 23: Locher, P. J., Stappers, P. J., and Overbeeke, K. The role of balance as an organizing design principle underlying adults' compositional strategies for creating visual displays. Acta Psychologica 99 (1998), 141--161.
• 24: Locher, P. J., Stappers, P. J., and Overbeeke, K. An empirical evaluation of the visual rightness theory of pictorial composition. Acta Psychologica 103 (1999), 261--280.
• 25: Simon Lok , Steven Feiner , Gary Ngai, Evaluation of visual balance for automated layout, Proceedings of the 9th international conference on Intelligent user interface, January 13-16, 2004, Funchal, Madeira, Portugal [doi>10.1145/964442.964462]
• 26: McManus, I., Edmondson, D., and Rodgers, J. Balance in pictures. British Journal of Psychology 76 (1985), 73--94.
• 27: Peterson, B. Learning to see Creatively: How to compose great photographs. AMPHOTO, 1988.
• 28: Setlur, V., Takagi, S., Raskar, R., Gleicher, M., and Gooch, B. Automatic image retargeting. ACM SIGGRAPH Technical Sketch.
• 29: Bongwon Suh , Haibin Ling , Benjamin B. Bederson , David W. Jacobs, Automatic thumbnail cropping and its effectiveness, Proceedings of the 16th annual ACM symposium on User interface software and technology, p.95-104, November 02-05, 2003, Vancouver, Canada [doi>10.1145/964696.964707]
• 30: Roel Vertegaal, The GAZE groupware system: mediating joint attention in multiparty communication and collaboration, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.294-301, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303065]
• 31: Roel Vertegaal, Designing attentive interfaces, Proceedings of the 2002 symposium on Eye tracking research & applications, March 25-27, 2002, New Orleans, Louisiana [doi>10.1145/507072.507077]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)

Keywords:
composition, cropping, evaluation, eye tracking, photography, visual perception