Automatic thumbnail cropping and its effectiveness

ABSTRACT

Thumbnail images provide users of image retrieval and browsing systems with a method for quickly scanning large numbers of images. Recognizing the objects in an image is important in many retrieval tasks, but thumbnails generated by shrinking the original image often render objects illegible. We study the ability of computer vision systems to detect key components of images so that automated cropping, prior to shrinking, can render objects more recognizable. We evaluate automatic cropping techniques 1) based on a general method that detects salient portions of images, and 2) based on automatic face detection. Our user study shows that these methods result in small thumbnails that are substantially more recognizable and easier to find in the context of visual search.

REFERENCES

• 1: 1. ACDSee, ACD Systems, http://www.adsystems.com
• 2: 2. Adobe Photoshop Album, Adobe Systems Inc., http://www.adobe.com/products/photoshopalbum/
• 3: Benjamin B. Bederson, PhotoMesa: a zoomable image browser using quantum treemaps and bubblemaps, Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida [doi>10.1145/502348.502359]
• 4: C. A. Burton , L. J. Johnston , E. A. Sonenberg, Case study: an empirical investigation of thumbnail image recognition, Proceedings of the 1995 IEEE Symposium on Information Visualization, p.115, October 30-31, 1995, Atlanta, Georgia
• 5: 5. Chen, L., Xie, X., Fan, X., Ma, W., Zhang, H., and Zhou, H. A Visual attention model for adapting images on small displays, ACM Multimedia Systems Journal, To appear in Fall 2003.
• 6: Xiangrong Chen , HongJiang Zhang, Text Area Detection from Video Frames, Proceedings of the Second IEEE Pacific Rim Conference on Multimedia: Advances in Multimedia Information Processing, p.222-228, October 24-26, 2001
• 7: 7. Corbis, http://www.corbis.com
• 8: Doug DeCarlo , Anthony Santella, Stylization and abstraction of photographs, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 9: 9. Face Detection Demonstration. Robotics Institute, Carnegie Mellon University http://www.vasc.ri.cmu.edu/cgi-bin/demos/findface.cgi
• 10: Myron Flickner , Harpreet Sawhney , Wayne Niblack , Jonathan Ashley , Qian Huang , Byron Dom , Monika Gorkani , Jim Hafner , Denis Lee , Dragutin Petkovic , David Steele , Peter Yanker, Query by Image and Video Content: The QBIC System, Computer, v.28 n.9, p.23-32, September 1995 [doi>10.1109/2.410146]
• 11: 11. Folk, C.L., Remington, R.W., and Johnston, J.C. Involuntary covert orienting is contingent on attentional control settings. Journal of Experimental Psychology: HP&P, 18:1030-44, 1992.
• 12: 12. Itti, L., and Koch, C. A Comparison of Feature Combination Strategies for Saliency-Based Visual Attention Systems, SPIE human vision and electronic imaging IV(HVEI'99), San Jose, CA, pp. 473-482, 1999.
• 13: Laurent Itti , Christof Koch , Ernst Niebur, A Model of Saliency-Based Visual Attention for Rapid Scene Analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.20 n.11, p.1254-1259, November 1998 [doi>10.1109/34.730558]
• 14: 14. Kang, H., and Shneiderman, B. Visualization Methods for Personal Photo Collections: Browsing and Searching in the PhotoFinder, In Proc. of IEEE International Conference on Multimedia and Expo (ICME2000) New York: IEEE, pp. 1539-1542, 2000.
• 15: Stan Z. Li , Long Zhu , ZhenQiu Zhang , Andrew Blake , HongJiang Zhang , Harry Shum, Statistical Learning of Multi-view Face Detection, Proceedings of the 7th European Conference on Computer Vision-Part IV, p.67-81, May 28-31, 2002
• 16: 16. Milanese, R., Wechsler H., Gil S., Bost J., and Pun T. Integration of Bottom-Up and Top-Down Cues for Visual Attention Using Non-Linear Relaxation, In proc. of Computer Vision and Pattern Recognition, IEEE, pp. 781-785, 1994.
• 17: 17. Milanese, R. Detecting Salient Regions in an Image: from Biological Evidence to Computer Implementation, Ph.D. thesis, Univ. of Geneva, 1993.
• 18: 18. Palmer, J. Ames, C.T., Lindsey, D.T. Measuring the Effect of Attention on Simple Visual Search. Journal of Experimental Psychology: Human Perception & Performance, 19, pp. 108-130, 1993.
• 19: 19. Schneiderman, H., and Kanade, T. A Statistical Model for 3D Object Detection Applied to Faces and Cars. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, June, 2000.
• 20: 20. Vimas Technologies. http://www.vimas.com
• 21: 21. Wolfe, J.M. Guided Search 2.0: A Revised Model of Visual Search, Psychonomic Buttletin and Review, Vol. 1, No. 2, pp. 202-238, 1994.
• 22: Ming-Hsuan Yang , David J. Kriegman , Narendra Ahuja, Detecting Faces in Images: A Survey, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.1, p.34-58, January 2002 [doi>10.1109/34.982883]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.5 Reconstruction

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.2 Design Tools and Techniques
          Subjects: User interfaces

  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

  I. Computing Methodologies
  I.6 SIMULATION AND MODELING
      I.6.8 Types of Simulation
          Subjects: Visual

General Terms:
Design, Experimentation

Keywords:
face detection, image cropping, saliency map, thumbnail, usability study, visual search, zoomable user interfaces