Learning user intention in relevance feedback using optimization

ABSTRACT

We present an optimization based approach to simultaneously extracting user interested objects from multiple relevance feedback images. We introduce a novel three-term cost unction; the first term measures the smoothness of local image regions within each individual image; the second term measures the homogeneity of user interested objects across different images; the third term favours the assumption that user interested objects will appear most frequently in the positive feedback examples. To model user interested regions in the query image and all multiple positive feedback images simultaneously, we employ a set of local image patch appearance prototypes to link image pixels across multiple images in order to reduce the complexity. Optimizing the cost function segments out the user interested objects from the query and all positive user feedback images simultaneously, which in turn enables the selection of relevant image features for refining image retrieval. We also present an optimization based manifold learning method which uses feedback samples as constraints to perform image retrieval. We present experimental results to demonstrate the effectiveness of our new methods.

REFERENCES

• 1: Blake, A., Rother, C., Brown, M., Perez, P. and Torr, P. Interactive image segmentation using an adaptive GMMRF model. In Proc ECCV'04
• 2: Borenstein, E. and Ullman, S. Learning to segment. In Proc. ECCV'04
• 3: Chad Carson , Serge Belongie , Hayit Greenspan , Jitendra Malik, Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.8, p.1026-1038, August 2002 [doi>10.1109/TPAMI.2002.1023800]
• 4: Hackbusch, W. Multi-grid Methods and Applications. Springer, Berlin, 1985.
• 5: Jing, F., Li, M., Zhang, H.J. and Zhang, B. Region-based relevance feedback in image retrieval. IEEE Trans. on Circuits and Systems for Video Technology. 14(5):672--681, 2004.
• 6: Minka, T.P. and Picard, R.W. Interactive Learning Using A Society of Models, Pattern Recognition, 30(4):565--581, 1997.
• 7: Qiu, G. and Guan, J. Color by Linear Neighborhood Embedding. In Proc. ICIP'05
• 8: Qiu, G. Indexing chromatic and achromatic patterns for content-based colour image retrieval. Pattern Recognition. 35:1675--1686, 2002.
• 9: Carsten Rother , Tom Minka , Andrew Blake , Vladimir Kolmogorov, Cosegmentation of Image Pairs by Histogram Matching - Incorporating a Global Constraint into MRFs, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.993-1000, June 17-22, 2006 [doi>10.1109/CVPR.2006.91]
• 10: Lawrence K. Saul , Sam T. Roweis, Think globally, fit locally: unsupervised learning of low dimensional manifolds, The Journal of Machine Learning Research, 4, p.119-155, 12/1/2003 [doi>10.1162/153244304322972667]
• 11: Jianbo Shi , Jitendra Malik, Normalized Cuts and Image Segmentation, Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97), p.731, June 17-19, 1997
• 12: Simon Tong , Edward Chang, Support vector machine active learning for image retrieval, Proceedings of the ninth ACM international conference on Multimedia, September 30-October 05, 2001, Ottawa, Canada [doi>10.1145/500141.500159]
• 13: Vasconcelos, N. and Lippman, A. Learning from user feedback in image retrieval system. In Proc. NIPS'99
• 14: James Z. Wang , Jia Li , Gio Wiederhold, SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.23 n.9, p.947-963, September 2001 [doi>10.1109/34.955109]
• 15: J. Winn , N. Jojic, LOCUS: Learning Object Classes with Unsupervised Segmentation, Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1, p.756-763, October 17-20, 2005 [doi>10.1109/ICCV.2005.148]
• 16: Zhou, X. S. and Huang, T. S. Relevance feedback in image retrieval: A comprehensive review. Multimedia Syst. 8(6):536--544, 2003.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Relevance feedback

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

General Terms:
Algorithms, Experimentation

Keywords:
image retrieval, image segmentation, relevance feedback, semi-supervised learning