Learning instance specific distances using metric propagation

ABSTRACT

In many real-world applications, such as image retrieval, it would be natural to measure the distances from one instance to others using instance specific distance which captures the distinctions from the perspective of the concerned instance. However, there is no complete framework for learning instance specific distances since existing methods are incapable of learning such distances for test instance and unlabeled data. In this paper, we propose the Isd method to address this issue. The key of Isd is metric propagation, that is, propagating and adapting metrics of individual labeled examples to individual unlabeled instances. We formulate the problem into a convex optimization framework and derive efficient solutions. Experiments show that Isd can effectively learn instance specific distances for labeled as well as unlabeled instances. The metric propagation scheme can also be used in other scenarios.

REFERENCES

• 1: Mikhail Belkin , Partha Niyogi , Vikas Sindhwani, Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples, The Journal of Machine Learning Research, 7, p.2399-2434, 12/1/2006
• 2: Blake, C., Keogh, E., & Merz, C. J. (1998). UCI repository of machine learning databases. {http://www.ics.uci.edu/~mlearn/MLRepository.html}.
• 3: Frome, A., Singer, Y., & Malik, J. (2006). Image retrieval and classification using local distance functions. In Adv. Neural. Inf. Process. Syst. 19, 417--424.
• 4: Frome, A., Singer, Y., Sha, F., & Malik, J. (2007). Learning globally-consistent local distance functions for shape-based image retrieval and classification. Proc. 11th Intl. Conf. Comp. Vision (pp. 1--8).
• 5: Goldberger, J., Roweis, S., Hinton, G., & Salakhutdinov, R. (2005). Neighbourhood components analysis. In Adv. Neural Inf. Process. Syst. 19, 513--520.
• 6: Kwok, J., & Tsang, I. (2003). Learning with idealized kernels. Proc. 20th Intl. Conf. Mach. Learn. (pp. 400--407).
• 7: Zhenguo Li , Jianzhuang Liu , Xiaoou Tang, Pairwise constraint propagation by semidefinite programming for semi-supervised classification, Proceedings of the 25th international conference on Machine learning, p.576-583, July 05-09, 2008, Helsinki, Finland [doi>10.1145/1390156.1390229]
• 8: Weinberger, K. Q., Blitzer, J., & Saul, L. K. (2005). Distance metric learning for large margin nearest neighbor classification. In Adv. Neural Inf. Process. Syst. 17, 1473--1480.
• 9: Xing, E. P., Ng, A. Y., Jordan, M. I., & Russell, S. (2002). Distance metric learning with application to clustering with side-information. In Adv. Neural Inf. Process. Syst. 14, 505--512.
• 10: Yang, L. (2006). Distance metric learning: A comprehensive survey. {http://www.cse.msu.edu/~yangliu1/frame_survey_v2.pdf}.
• 11: Ruofei Zhang , Zhongfei (Mark) Zhang , Mingjing Li , Wei-Ying Ma , Hong-Jiang Zhang, A Probabilistic Semantic Model for Image Annotation and Multi-Modal Image Retrieva, Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1, p.846-851, October 17-20, 2005 [doi>10.1109/ICCV.2005.16]
• 12: Wei Zhang , Xiangyang Xue , Zichen Sun , Yue-Fei Guo , Hong Lu, Optimal dimensionality of metric space for classification, Proceedings of the 24th international conference on Machine learning, p.1135-1142, June 20-24, 2007, Corvalis, Oregon [doi>10.1145/1273496.1273639]
• 13: Zhou, D., Bousquet, O., Lal, T. N., Weston, J., & Schöölkopf, B. (2003). Learning with local and global consistency. In Adv. Neural Inf. Process. Syst. 17, 321--328.
• 14: Zhi-Hua Zhou , Hong-Bin Dai, Query-Sensitive Similarity Measure for Content-Based Image Retrieval, Proceedings of the Sixth International Conference on Data Mining, p.1211-1215, December 18-22, 2006 [doi>10.1109/ICDM.2006.131]
• 15: Zhou, Z.-H., & Yang, Y. (2005). Ensembling local learners through multimodal perturbation. IEEE Trans. Syst., Man and Cybern. - B, 35, 725--735.
• 16: Zhu, X., Ghahramani, Z., & Lafferty, J. (2003). Semi-supervised learning using Gaussian fields and harmonic functions. Proc. 20th Intl. Conf. Mach. Learn. (pp. 912--919).

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning

Additional Classification:
  G. Mathematics of Computing
  G.1 NUMERICAL ANALYSIS
      G.1.6 Optimization
          Subjects: Convex programming

  I. Computing Methodologies
  I.6 SIMULATION AND MODELING
      I.6.4 Model Validation and Analysis
      I.6.5 Model Development
          Subjects: Modeling methodologies

General Terms:
Algorithms, Design, Performance, Theory