A latent variable model for query expansion using the hidden markov model

ABSTRACT

We propose a novel probabilistic method based on the Hidden Markov Model(HMM) to learn the structure of a Latent Variable Model (LVM) for query language modeling. In the proposed LVM, the combinations of query terms are viewed as the latent variables and the segmented chunks from the feedback documents are used as the observations given these latent variables. Our extensive experiments shows that our method significantly outperforms a number of strong baselines in terms of both effectiveness and robustness.

REFERENCES

• 1: Jing Bai , Dawei Song , Peter Bruza , Jian-Yun Nie , Guihong Cao, Query expansion using term relationships in language models for information retrieval, Proceedings of the 14th ACM international conference on Information and knowledge management, October 31-November 05, 2005, Bremen, Germany [doi>10.1145/1099554.1099725]
• 2: Victor Lavrenko , W. Bruce Croft, Relevance based language models, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.120-127, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.383972]
• 3: Victor Lavrenko , James Allan , Edward DeGuzman , Daniel LaFlamme , Veera Pollard , Stephen Thomas, Relevance models for topic detection and tracking, Proceedings of the second international conference on Human Language Technology Research, March 24-27, 2002, San Diego, California
• 4: D. Song , P. D. Bruza, Towards context sensitive information inference, Journal of the American Society for Information Science and Technology, v.54 n.4, p.321-334, February 15, 2003 [doi>10.1002/asi.10213]
• 5: D. Song, Q. Huang, S. Rueger, and P. Bruza. Facilitating query decomposition in query language modeling by association rule mining using multiple sliding windows. In Proceedings of ECIR'2008.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

General Terms:
Algorithms

Keywords:
hidden markov model, information retrieval, latent variable model