Learning to rank using gradient descent

ABSTRACT

We investigate using gradient descent methods for learning ranking functions; we propose a simple probabilistic cost function, and we introduce RankNet, an implementation of these ideas using a neural network to model the underlying ranking function. We present test results on toy data and on data from a commercial internet search engine.

REFERENCES

• 1: Baum, E., & Wilczek, F. (1988). Supervised learning of probability distributions by neural networks. Neural Information Processing Systems (pp. 52--61).
• 2: Bradley, R., & Terry, M. (1952). The Rank Analysis of Incomplete Block Designs 1: The Method of Paired Comparisons. Biometrika, 39, 324--245.
• 3: Bromley, J., Bentz, J. W., Bottou, L., Guyon, I., LeCun, Y., Moore, C., Sackinger, E., & Shah, R. (1993). Signature Verification Using a "Siamese" Time Delay Neural Network. Advances in Pattern Recognition Systems using Neural Network Technologies, World Scientific (pp. 25--44)
• 4: Burges, C. (1996). Simplified support vector decision rules. Proc. International Conference on Machine Learning (ICML) 13 (pp. 71--77).
• 5: Caruana, R., Baluja, S., & Mitchell, T. (1996). Using the future to "sort out" the present: Rankprop and multitask learning for medical risk evaluation. Advances in Neural Information Processing Systems (NIPS) 8 (pp. 959--965).
• 6: Crammer, K., & Singer, Y. (2002). Pranking with ranking. NIPS 14.
• 7: Dekel, O., Manning, C., & Singer, Y. (2004). Loglinear models for label-ranking. NIPS 16.
• 8: Yoav Freund , Raj Iyer , Robert E. Schapire , Yoram Singer, An efficient boosting algorithm for combining preferences, The Journal of Machine Learning Research, 4, p.933-969, 12/1/2003
• 9: Harrington, E. (2003). Online ranking/collaborative filtering using the Perceptron algorithm. ICML 20.
• 10: Hastie, T., & Tibshirani, R. (1998). Classification by pairwise coupling. NIPS 10.
• 11: Herbrich, R., Graepel, T., & Obermayer, K. (2000). Large margin rank boundaries for ordinal regression. Advances in Large Margin Classifiers, MIT Press (pp. 115--132).
• 12: Kalervo Järvelin , Jaana Kekäläinen, IR evaluation methods for retrieving highly relevant documents, Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.41-48, July 24-28, 2000, Athens, Greece [doi>10.1145/345508.345545]
• 13: Kimeldorf, G. S., & Wahba, G. (1971). Some results on Tchebycheffian Spline Functions. J. Mathematical Analysis and Applications, 33, 82--95.
• 14: Yann LeCun , Léon Bottou , Genevieve B. Orr , Klaus-Robert Müller, Effiicient BackProp, Neural Networks: Tricks of the Trade, this book is an outgrowth of a 1996 NIPS workshop, p.9-50, January 1998
• 15: Mason, L., Baxter, J., Bartlett, P., & Frean, M. (2000). Boosting algorithms as gradient descent. NIPS 12 (pp. 512--518).
• 16: Thomas M. Mitchell, Machine Learning, McGraw-Hill Higher Education, 1997
• 17: Refregier, P., & Vallet, F. (1991). Probabilistic approaches for multiclass classification with neural networks. International Conference on Artificial Neural Networks (pp. 1003--1006).
• 18: Schölkopf, B., & Smola, A. (2002). Learning with kernels. MIT Press.