Automatic image retargeting

ABSTRACT

We present a non-photorealistic algorithm for retargeting large images to small size displays, particularly on mobile devices. This method adapts large images so that important objects in the image are still recognizable when displayed at a lower target resolution. Existing image manipulation techniques such as cropping works well for images containing a single important object, and down-sampling works well for images containing low frequency information. However, when these techniques are automatically applied to images with multiple objects, the image quality degrades and important information may be lost. Our algorithm addresses the case of multiple important objects in an image. The retargeting algorithm segments an image into regions, identifies important regions, removes them, fills the resulting gaps, resizes the remaining image, and re-inserts the important regions. Our approach lies in constructing a topologically constrained epitome of an image based on a visual attention model that is both comprehensible and size varying, making the method suitable for display-critical applications.

REFERENCES

• 1: http://www.adobe.com/. Adobe Photoshop.
• 2: Bregler, C., 1998. Tracking people with twists and exponential maps.
• 3: M. S. T. Carpendale , Catherine Montagnese, A framework for unifying presentation space, Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida [doi>10.1145/502348.502358]
• 4: Chen, L., Xie, X., Fan, X., Ma, W.-Y., Zhang, H.-J., and Zhou, H. 2002. A visual attention model for adapting images on small displays. Tech. Rep. MSR-TR-2002-125, Microsoft Research, Nov.
• 5: Chen, L.-Q., Xie, X., Fan, X., Ma, W.-Y., Zhang, H.-J., and Zhou, H.-Q. 2003. A visual attention model for adapting images on small displays. ACM Multimedia Systems Journal, 353--364.
• 6: Chen, L.-Q., Xie, X., Ma, W.-Y., AND Zhou, H.-Q. 2003. Image adaptation based on attention model for small-form factor devices. In Proceeding of the 9th International Conference on Multimedia Modeling, IEEE, 421--439.
• 7: Iddo Drori , Daniel Cohen-Or , Hezy Yeshurun, Fragment-based image completion, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003
• 8: Xin Fan , Xing Xie , He-Qin Zhou , Wei-Ying Ma, Looking into video frames on small displays, Proceedings of the eleventh ACM international conference on Multimedia, November 02-08, 2003, Berkeley, CA, USA [doi>10.1145/957013.957063]
• 9: http://www.gimp.org/. The GNU Image Manipulation Program.
• 10: Harrison, P. 2001. A non-hierarchical procedure for re-synthesis of complex textures. In Proceedings of WSCG 2001, 190--197.
• 11: Laurent Itti , Christof Koch , Ernst Niebur, A Model of Saliency-Based Visual Attention for Rapid Scene Analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.20 n.11, p.1254-1259, November 1998 [doi>10.1109/34.730558]
• 12: Nebojsa Jojic , Brendan J. Frey , Anitha Kannan, Epitomic analysis of appearance and shape, Proceedings of the Ninth IEEE International Conference on Computer Vision, p.34, October 13-16, 2003
• 13: Stephen Michael Kosslyn, On the ontological status of visual mental images, Proceedings of the 1978 workshop on Theoretical issues in natural language processing, July 25-27, 1978, Urbana-Champaign, Illinois [doi>10.3115/980262.980289]
• 14: Hao Liu , Xing Xie , Wei-Ying Ma , Hong-Jiang Zhang, Automatic browsing of large pictures on mobile devices, Proceedings of the eleventh ACM international conference on Multimedia, November 02-08, 2003, Berkeley, CA, USA [doi>10.1145/957013.957045]
• 15: Yu-Fei Ma , Hong-Jiang Zhang, Contrast-based image attention analysis by using fuzzy growing, Proceedings of the eleventh ACM international conference on Multimedia, November 02-08, 2003, Berkeley, CA, USA [doi>10.1145/957013.957094]
• 16: Yu-Fei Ma , Lie Lu , Hong-Jiang Zhang , Mingjing Li, A user attention model for video summarization, Proceedings of the tenth ACM international conference on Multimedia, December 01-06, 2002, Juan-les-Pins, France [doi>10.1145/641007.641116]
• 17: May, J. 2000. Perceptual principles and computer graphics. In Computer Graphics Forum, vol. 19.
• 18: P. Meer , B. Georgescu, Edge Detection with Embedded Confidence, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.23 n.12, p.1351-1365, December 2001 [doi>10.1109/34.977560]
• 19: Niblack, W., Barber, R., Equitz, W., Flickner, M., Glasman, E. H., Petkovic, D., Yanker, P., Faloutsos, C., and Taubin, G. 1993. The qbic project: Querying images by content, using color, texture, and shape. In Vol. 1908, SPIE, 173--187.
• 20: Patrick Pérez , Michel Gangnet , Andrew Blake, Poisson image editing, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003
• 21: Rogers, S. 1995. Perceiving pictorial space. In Perception of Space and Motion, Chapter 4. Academic Press, Inc.
• 22: Rowley, H. A., Baluja, S., and Kanade, T. 1996. Human face detection in visual scenes. In Advances in Neural Information Processing Systems, The MIT Press, D. S. Touretzky, M. C. Mozer, and M. E. Hasselmo, Eds., vol. 8, 875--881.
• 23: Holly Rushmeier , Harrison Barrett , Penny Rheingans , Sam Uselton , Andrew Watson, Perceptual measures for effective visualizations, Proceedings of the 8th conference on Visualization '97, p.515-517, October 18-24, 1997, Phoenix, Arizona, United States
• 24: Schneiderman, H., and Kanade, T. 2000. A statistical model for 3d object detection applied to faces and cars. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition - Volume 1, IEEE, 746--751.
• 25: Bongwon Suh , Haibin Ling , Benjamin B. Bederson , David W. Jacobs, Automatic thumbnail cropping and its effectiveness, Proceedings of the 16th annual ACM symposium on User interface software and technology, p.95-104, November 02-05, 2003, Vancouver, Canada [doi>10.1145/964696.964707]
• 26: Michael J. Swain , Dana H. Ballard, Color indexing, International Journal of Computer Vision, v.7 n.1, p.11-32, Nov. 1991 [doi>10.1007/BF00130487]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.3 Picture/Image Generation
          Subjects: Display algorithms

Keywords:
image adaptation, image attention, image cropping, non-photorealistic rendering, small displays, visual perception