A unified framework for semantics and feature based relevance feedback in image retrieval systems

ABSTRACT

The relevance feedback approach to image retrieval is a powerful technique and has been an active research direction for the past few years. Various ad hoc parameter estimation techniques have been proposed for relevance feedback. In addition, methods that perform optimization on multi-level image content model have been formulated. However, these methods only perform relevance feedback on the low-level image features and fail to address the images' semantic content. In this paper, we propose a relevance feedback technique, iFind, to take advantage of the semantic contents of the images in addition to the low-level features. By forming a semantic network on top of the keyword association on the images, we are able to accurately deduce and utilize the images' semantic contents for retrieval purposes. The accuracy and effectiveness of our method is demonstrated with experimental results on real-world image collections.

REFERENCES

• 1: Chris Buckley , Gerard Salton, Optimization of relevance feedback weights, Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval, p.351-357, July 09-13, 1995, Seattle, Washington, United States [doi>10.1145/215206.215383]
• 2: Cox, I.J., Miller, M.L., Minka, T.P., Papathornas, T.V., Yianilos, P.N. "The Bayesian Image Retrieval System, PicHunter: Theory, Implementation, and Psychophysical Experiments" IEEE Tran. On Image Processing, Volume 9, Issue 1, pp. 20-37, Jan. 2000.
• 3: Yoshiharu Ishikawa , Ravishankar Subramanya , Christos Faloutsos, MindReader: Querying Databases Through Multiple Examples, Proceedings of the 24rd International Conference on Very Large Data Bases, p.218-227, August 24-27, 1998
• 4: Lee, C., Ma, W. Y., and Zhang, H. J. "Information Embedding Based on user's relevance Feedback for Image Retrieval," Technical Report HP Labs, 1998.
• 5: Paek S., Sable C.L., Hatzivassiloglou V., Jaimes A.,Schiffman B.H., Chang S. F., Mckeown K.R, "Integration of Visual and Text-Based Approaches for the Content Labeling and Classification of Photographs", SIGIR'99.
• 6: Rui, Y., Huang, T. S., and Mehrotra, S. "Content-Based Image Retrieval with Relevance Feedback in MARS," in Proc. IEEE Int. Conf. on Image proc., 1997.
• 7: Yong Rui , Thomas S. Huang, A novel relevance feedback technique in image retrieval, Proceedings of the seventh ACM international conference on Multimedia (Part 2), p.67-70, October 30-November 05, 1999, Orlando, Florida, United States [doi>10.1145/319878.319896]
• 8: Gerard Salton , Michael J. McGill, Introduction to Modern Information Retrieval, McGraw-Hill, Inc., New York, NY, 1986
• 9: W. M. Shaw, Jr., Term-relevance computations and perfect retrieval performance, Information Processing and Management: an International Journal, v.31 n.4, p.491-498, July 1995 [doi>10.1016/0306-4573(95)00011-5]

INDEX TERMS

Primary Classification:
  F. Theory of Computation
  F.3 LOGICS AND MEANINGS OF PROGRAMS

Additional Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.4 Systems
          Subjects: Multimedia databases
      H.2.8 Database applications
          Subjects: Image databases
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.4 Knowledge Representation Formalisms and Methods
          Subjects: Semantic networks
  I.5 PATTERN RECOGNITION
      I.5.2 Design Methodology
          Subjects: Feature evaluation and selection

General Terms:
Design, Experimentation, Languages, Management, Measurement, Performance, Theory

Keywords:
image retrieval, image semantics, multimedia database, relevance feedback