Take a look at the new version of this page: [ beta version ]. Tell us what you think.

ABSTRACT

Eye tracking experiments have shown that titles of Web search results play a crucial role in guiding a user's search process. We present a machine-learned algorithm that trains a boosted tree to pick the most relevant title for a Web search result. We compare two modeling approaches: i) using absolute editorial judgments and ii) using pairwise preference judgments. We find that the pairwise modeling approach gives better results in terms of three offline metrics. We present results of our models in four regions. We also describe a hybrid user satisfaction evaluation process -- search success -- that combines page relevance and user click behavior, and show that our machine-learned algorithm improves in search success.

REFERENCES

• 1: Charles L. A. Clarke , Eugene Agichtein , Susan Dumais , Ryen W. White, The influence of caption features on clickthrough patterns in web search, Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, July 23-27, 2007, Amsterdam, The Netherlands [doi>10.1145/1277741.1277767]
• 2: S. Dumais, T. Joachims, K. Bharat, and A. Weigend. SIGIR 2003 workshop report: Implicit measures of user interests and preferences. ACM SIGIR Forum, 37:50--54, 2003.
• 3: Jerome H. Friedman, Stochastic gradient boosting, Computational Statistics & Data Analysis, v.38 n.4, p.367-378, 28 February 2002 [doi>10.1016/S0167-9473(01)00065-2]
• 4: M. A. Hearst. Models of information seeking. In Search User Interfaces. 2009.
• 5: R. Khan, D. Mease, and R. Patel. The impact of result abstracts on task completion time. In Proc. of WWW Workshop on Web Search Result Summarization and Presentation, 2009.
• 6: Julian Kupiec , Jan Pedersen , Francine Chen, A trainable document summarizer, Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval, p.68-73, July 09-13, 1995, Seattle, Washington, United States [doi>10.1145/215206.215333]
• 7: P. Li, C. J. Burges, and Q. Wu. Mcrank: Learning to rank using multiple classification and gradient boosting. In Proc. 21st Proc. of Advances in Neural Information Processing Systems, 2007.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Selection process

General Terms:
Algorithms, Experimentation, Theory

Keywords:
machine learning, user satisfaction, web summarization