Relevance aggregation projections for image retrieval

ABSTRACT

To narrow the semantic gap in content-based image retrieval (CBIR), relevance feedback is utilized to explore knowledge about the user's intention in finding a target image or a image category. Users provide feedback by marking images returned in response to a query image as relevant or irrelevant. Existing research explores such feedback to refine querying process, select features, or learn a image classifier. However, the vast amount of unlabeled images is ignored and often substantially limited examples are engaged into learning. In this paper, we address the two issues and propose a novel effective method called Relevance Aggregation Projections (RAP) for learning potent subspace projections in a semi-supervised way. Given relevances and irrelevances specified in the feedback, RAP produces a subspace within which the relevant examples are aggregated into a single point and the irrelevant examples are simultaneously separated by a large margin. Regarding the query plus its feedback samples as labeled data and the remainder as unlabeled data, RAP falls in a special paradigm of imbalanced semi-supervised learning. Through coupling the idea of relevance aggregation with semi-supervised learning, we formulate a constrained quadratic optimization problem to learn the subspace projections which entail semantic mining and therefore make the underlying CBIR system respond to the user's interest accurately and promptly. Experiments conducted over a large generic image database show that our subspace approach outperforms existing subspace methods for CBIR even with few iterations of user feedback.

REFERENCES

• 1: Mikhail Belkin , Partha Niyogi, Laplacian Eigenmaps for dimensionality reduction and data representation, Neural Computation, v.15 n.6, p.1373-1396, June 2003 [doi>10.1162/089976603321780317]
• 2: Mikhail Belkin , Partha Niyogi , Vikas Sindhwani, Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples, The Journal of Machine Learning Research, 7, p.2399-2434, 12/1/2006
• 3: Deng Cai , Xiaofei He , Jiawei Han, Spectral regression: a unified subspace learning framework for content-based image retrieval, Proceedings of the 15th international conference on Multimedia, September 25-29, 2007, Augsburg, Germany [doi>10.1145/1291233.1291329]
• 4: O. Chapelle, B. Schölkopf, and A. Zien. Semi-Supervised Learning. MIT Press, Cambridge, MA, 2006.
• 5: Hwann-Tzong Chen , Huang-Wei Chang , Tyng-Luh Liu, Local Discriminant Embedding and Its Variants, Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2, p.846-853, June 20-26, 2005 [doi>10.1109/CVPR.2005.216]
• 6: F. Chung. Spectral graph theory. In CBMS Regional Conference Series in Mathematics, American Mathematical Society, number 92, 1997.
• 7: T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer-Verlag, New York, 2001.
• 8: X. He and P. Niyogi. Locality preserving projections. In Advances in Neural Information Processing Systems 16, MIT Press, Cambridge, MA, 2004.
• 9: Xiaofei He , Shuicheng Yan , Yuxiao Hu , Partha Niyogi , Hong-Jiang Zhang, Face Recognition Using Laplacianfaces, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.27 n.3, p.328-340, March 2005 [doi>10.1109/TPAMI.2005.55]
• 10: Steven C. H. Hoi , Michael R. Lyu , Rong Jin, A Unified Log-Based Relevance Feedback Scheme for Image Retrieval, IEEE Transactions on Knowledge and Data Engineering, v.18 n.4, p.509-524, April 2006 [doi>10.1109/TKDE.2006.53]
• 11: Steven C. H. Hoi , Wei Liu , Michael R. Lyu , Wei-Ying Ma, Learning Distance Metrics with Contextual Constraints for Image Retrieval, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.2072-2078, June 17-22, 2006 [doi>10.1109/CVPR.2006.167]
• 12: W. Jiang, G. Er, Q. Dai, and J. Gu. Similarity-based online feature selection in content-based image retrieval. IEEE Trans. on Image Processing, 15(3):702--711, 2006.
• 13: Yen-Yu Lin , Tyng-Luh Liu , Hwann-Tzong Chen, Semantic manifold learning for image retrieval, Proceedings of the 13th annual ACM international conference on Multimedia, November 06-11, 2005, Hilton, Singapore [doi>10.1145/1101149.1101193]
• 14: S. Roweis and L. Saul. Nonlinear dimensionality reduction by locally linear embedding. Science, 290(5500):2323--2326, 2000.
• 15: Y. Rui, T. S. Huang, M. Ortega, and S. Mehrotra. Relevance feedback: A powerful tool in interactive content-based image retrieval. IEEE Trans. on Circuits and Systems for Video Technology, 8(5):644--655, 1998.
• 16: B. Schölkopf and A. Smola. Learning with Kernels. MIT Press, Cambridge, MA, 2002.
• 17: Arnold W. M. Smeulders , Marcel Worring , Simone Santini , Amarnath Gupta , Ramesh Jain, Content-Based Image Retrieval at the End of the Early Years, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.22 n.12, p.1349-1380, December 2000 [doi>10.1109/34.895972]
• 18: J. B. Tenenbaum, V. de Silva, and J. C. Langford. A global geometric framework for nonlinear dimensionality reduction. Science, 290(5500):2319--2323, 2000.
• 19: Simon Tong , Edward Chang, Support vector machine active learning for image retrieval, Proceedings of the ninth ACM international conference on Multimedia, September 30-October 05, 2001, Ottawa, Canada [doi>10.1145/500141.500159]
• 20: Jie Yu , Qi Tian, Learning image manifolds by semantic subspace projection, Proceedings of the 14th annual ACM international conference on Multimedia, October 23-27, 2006, Santa Barbara, CA, USA [doi>10.1145/1180639.1180710]
• 21: D. Zhou, O. Bousquet, T. Lal, J. Weston, and B. Schölkopf. Learning with local and global consistency. In Advances in Neural Information Processing Systems 16, MIT Press, Cambridge, MA, 2004.
• 22: X. Zhu, Z. Ghahramani, and J. Lafferty. Semi-supervised learning using gaussian fields and harmonic functions. In Proc. International Conference on Machine Learning, 2003.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Relevance feedback

General Terms:
Algorithms, Experimentation, Measurement, Performance, Theory

Keywords:
dimensionality reduction, image retrieval, relevance aggregation projections, relevance feedback, semi-supervised learning, subspace learning