Impediments to general purpose Content Based Image search

ABSTRACT

Challenges faced by prevailing text metadata paradigms for online image search have inspired overwhelming research in Content Based Image Retrieval (CBIR). A multitude of approaches have been introduced within the literature, yet relatively few image search engines have been made publicly available on the web. Aside from challenges facing the user, such as describing a visual query using keywords, or finding an appropriate example image to initiate a visual search, all systems must inevitably grapple with the sensory and semantic gaps [Smeulders et al. 2000], which essentially represent a loss of information in the abstraction process. In this work, we challenge commonly suggested approaches to improving CBIR and illustrate drawbacks of relying on textual data, as well as visual data, in general CBIR search. We provide cogent examples using online visual search engines Behold™, Tiltomo Beta, Pixilimar, and Riya™ Beta. These examples demonstrate the effect of semantic ambiguities in natural language, which extend to search terms and text tags.

REFERENCES

• 1: Jonathan Ashley , Myron Flickner , James Hafner , Denis Lee , Wayne Niblack , Dragutin Petkovic, The query by image content (QBIC) system, Proceedings of the 1995 ACM SIGMOD international conference on Management of data, p.475, May 22-25, 1995, San Jose, California, United States
• 2: Ritendra Datta , Dhiraj Joshi , Jia Li , James Z. Wang, Image retrieval: Ideas, influences, and trends of the new age, ACM Computing Surveys (CSUR), v.40 n.2, p.1-60, April 2008 [doi>10.1145/1348246.1348248]
• 3: Ritendra Datta , Jia Li , James Z. Wang, Content-based image retrieval: approaches and trends of the new age, Proceedings of the 7th ACM SIGMM international workshop on Multimedia information retrieval, November 10-11, 2005, Hilton, Singapore [doi>10.1145/1101826.1101866]
• 4: Nicolas Hervé , Nozha Boujemaa, Image annotation: which approach for realistic databases?, Proceedings of the 6th ACM international conference on Image and video retrieval, p.170-177, July 09-11, 2007, Amsterdam, The Netherlands [doi>10.1145/1282280.1282310]
• 5: M. L. Kherfi , D. Ziou , A. Bernardi, Image Retrieval from the World Wide Web: Issues, Techniques, and Systems, ACM Computing Surveys (CSUR), v.36 n.1, p.35-67, March 2004 [doi>10.1145/1013208.1013210]
• 6: Lampert, A. 2006. The broken promise of automatic image tagging, November 2006. Available online: http://www.sgi.nu/diary/2006/11/16/the-broken-promise-of-automatic-image-tagging/. Accessed January 08, 2009.
• 7: Li, J., Gray, R. M., and Olshen, R. A. 2000. Multiresolution image classification by hierarchical modeling with two-dimensional hidden markov models. IEEE Transactions on Information Theory, 46, 5 (Aug. 2000), 1826--1841. DOI= 10.1109/18.857794
• 8: Jia Li , James Z. Wang, Real-time computerized annotation of pictures, Proceedings of the 14th annual ACM international conference on Multimedia, October 23-27, 2006, Santa Barbara, CA, USA [doi>10.1145/1180639.1180841]
• 9: David G. Lowe, Object Recognition from Local Scale-Invariant Features, Proceedings of the International Conference on Computer Vision-Volume 2, p.1150, September 20-25, 1999
• 10: Pavlidis, T. 2009. Limitations of content based image retrieval, June 2008. Available online: http://www.theopavlidis.com/technology/CBIR/PaperB/Apr08.htm. Accessed January 08, 2009.
• 11: Popescu, A. and Kanellos, J. 2008. Multilingual and content based access to flickr images. In Proceedings of the 3rd International Conference on Information and Communication Technologies: From Theory to Applications. (April 2008). ICTTA'08. DOI= 10.1109/ICTTA.2008.4530012
• 12: Arnold W. M. Smeulders , Marcel Worring , Simone Santini , Amarnath Gupta , Ramesh Jain, Content-Based Image Retrieval at the End of the Early Years, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.22 n.12, p.1349-1380, December 2000 [doi>10.1109/34.895972]
• 13: Joel Spolsky. The law of leaky abstractions, November 2002. Available online: http://www.joelonsoftware.com/articles/LeakyAbstractions.html. Accessed January 08, 2009.
• 14: Remco C. Veltkamp and Mirela Tanasa. Content-based image retrieval systems: A survey. Technical report, Dept. of Computing Science, Utrect University, 2000.
• 15: James Ze Wang , Gio Wiederhold , Oscar Firschein, System for Screening Objectionable Images Using Daubechies' Wavelets and Color Histograms, Proceedings of the 4th International Workshop on Interactive Distributed Multimedia Systems and Telecommunication Services, p.20-30, September 10-12, 1997
• 16: Alexei Yavlinsky , Daniel Heesch, An online system for gathering image similarity judgements, Proceedings of the 15th international conference on Multimedia, September 25-29, 2007, Augsburg, Germany [doi>10.1145/1291233.1291372]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis
          Subjects: Color

Additional Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis
          Subjects: Object recognition
  I.5 PATTERN RECOGNITION
      I.5.2 Design Methodology
          Subjects: Feature evaluation and selection; Classifier design and evaluation

General Terms:
Algorithms

Keywords:
computer vision, image analysis, image processing, information retrieval, semantic gap, visual similarity