Visual diversification of image search results

ABSTRACT

Due to the reliance on the textual information associated with an image, image search engines on the Web lack the discriminative power to deliver visually diverse search results. The textual descriptions are key to retrieve relevant results for a given user query, but at the same time provide little information about the rich image content.

In this paper we investigate three methods for visual diversification of image search results. The methods deploy lightweight clustering techniques in combination with a dynamic weighting function of the visual features, to best capture the discriminative aspects of the resulting set of images that is retrieved. A representative image is selected from each cluster, which together form a diverse result set.

Based on a performance evaluation we find that the outcome of the methods closely resembles human perception of diversity, which was established in an extensive clustering experiment carried out by human assessors.

REFERENCES

• 1: A. Bhattacharyya. On a measure of divergence between two statistical populations defined by probability distribution. Bul l. Calcutta Math. Soc., 35:99--109, 1493.
• 2: Deng Cai , Xiaofei He , Zhiwei Li , Wei-Ying Ma , Ji-Rong Wen, Hierarchical clustering of WWW image search results using visual, textual and link information, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027747]
• 3: S. A. Chatzichristo's and Y. S. Boutalis. Cedd: Color and edge directivity descriptor: A compact descriptor for image indexing and retrieval. In A. Gasteratos, M. Vincze, and J. K. Tsotsos, editors, ICVS, volume 5008 of Lecture Notes in Computer Science, pages 312--322. Springer, 2008.
• 4: Ritendra Datta , Dhiraj Joshi , Jia Li , James Z. Wang, Image retrieval: Ideas, influences, and trends of the new age, ACM Computing Surveys (CSUR), v.40 n.2, p.1-60, April 2008 [doi>10.1145/1348246.1348248]
• 5: E. Fowlkes and C. Mallows. A method for comparing two hierarchical clusterings. Journal of the American Statistical Association, 78(383):553--569, 1983.
• 6: J. Goldberger, S. Gordon, and H. Greenspan. Unsupervised image-set clustering using an information theoretic framework. IEEE Transactions on Image Processing, 15(2):449--458, 2006.
• 7: Jing Huang , S. Ravi Kumar , Ramin Zabih, An automatic hierarchical image classification scheme, Proceedings of the sixth ACM international conference on Multimedia, p.219-228, September 13-16, 1998, Bristol, United Kingdom [doi>10.1145/290747.290774]
• 8: Ying Liu , Dengsheng Zhang , Guojun Lu , Wei-Ying Ma, A survey of content-based image retrieval with high-level semantics, Pattern Recognition, v.40 n.1, p.262-282, January, 2007 [doi>10.1016/j.patcog.2006.04.045]
• 9: Marina Meilă, Comparing clusterings---an information based distance, Journal of Multivariate Analysis, v.98 n.5, p.873-895, May, 2007 [doi>10.1016/j.jmva.2006.11.013]
• 10: Pierre-Alain Moëllic , Jean-Emmanuel Haugeard , Guillaume Pitel, Image clustering based on a shared nearest neighbors approach for tagged collections, Proceedings of the 2008 international conference on Content-based image and video retrieval, July 07-09, 2008, Niagara Falls, Canada [doi>10.1145/1386352.1386390]
• 11: B.S. Manjunath , Phillipe Salembier , Thomas Sikora, Introduction to MPEG-7: Multimedia Content Description Interface, John Wiley & Sons, Inc., New York, NY, 2002
• 12: Arnold W. M. Smeulders , Marcel Worring , Simone Santini , Amarnath Gupta , Ramesh Jain, Content-Based Image Retrieval at the End of the Early Years, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.22 n.12, p.1349-1380, December 2000 [doi>10.1109/34.895972]
• 13: Kai Song , Yonghong Tian , Wen Gao , Tiejun Huang, Diversifying the image retrieval results, Proceedings of the 14th annual ACM international conference on Multimedia, October 23-27, 2006, Santa Barbara, CA, USA [doi>10.1145/1180639.1180789]
• 14: H. Tamura, S. Mori, and T. Yamawaki. Texture features corresponding to visual perception. IEEE Transactions on Systems, Man and Cybernetics, 8(6), 1978.
• 15: R. van Zwol. Multimedia strategies for b3-sdr, based on principle component analysis. In Advances in XML Information Retrieval, Lecture Notes in Computer Science. Springer, 2006.
• 16: Roelof van Zwol , Vanessa Murdock , Lluis Garcia Pueyo , Georgina Ramirez, Diversifying image search with user generated content, Proceeding of the 1st ACM international conference on Multimedia information retrieval, October 30-31, 2008, Vancouver, British Columbia, Canada [doi>10.1145/1460096.1460109]
• 17: D. Wallace. Comment. Journal of the American Statistical Association, 78(383):569--576, 1983.
• 18: Shuo Wang , Feng Jing , Jibo He , Qixing Du , Lei Zhang, IGroup: presenting web image search results in semantic clusters, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240718]
• 19: Kilian Quirin Weinberger , Malcolm Slaney , Roelof Van Zwol, Resolving tag ambiguity, Proceeding of the 16th ACM international conference on Multimedia, October 26-31, 2008, Vancouver, British Columbia, Canada [doi>10.1145/1459359.1459375]
• 20: Peter Wilkins , Paul Ferguson , Alan F. Smeaton, Using score distributions for query-time fusion in multimediaretrieval, Proceedings of the 8th ACM international workshop on Multimedia information retrieval, October 26-27, 2006, Santa Barbara, California, USA [doi>10.1145/1178677.1178687]
• 21: the 8th ACM international workshop on Multimedia information retrieval, pages 51--60, New York, NY, USA, 2006. ACM.
• 22: S. A. Yahia, P. Bhat, J. Shanmugasundaram, U. Srivastava, and E. Vee. Efficient online computation of diverse query results. In Proceedings of VLDB, 2007.
• 23: Benyu Zhang , Hua Li , Yi Liu , Lei Ji , Wensi Xi , Weiguo Fan , Zheng Chen , Wei-Ying Ma, Improving web search results using affinity graph, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, August 15-19, 2005, Salvador, Brazil [doi>10.1145/1076034.1076120]
• 24: and development in information retrieval, pages 504--511, New York, NY, USA, 2005. ACM.
• 25: Cai-Nicolas Ziegler , Sean M. McNee , Joseph A. Konstan , Georg Lausen, Improving recommendation lists through topic diversification, Proceedings of the 14th international conference on World Wide Web, May 10-14, 2005, Chiba, Japan [doi>10.1145/1060745.1060754]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.5 On-line Information Services

General Terms:
Algorithms, Experimentation, Performance

Keywords:
flickr, image clustering, visual diversity