A proposal for chemical information retrieval evaluation

ABSTRACT

Based on the important progresses made in information retrieval (IR) in terms of theoretical models and evaluations, more and more attention has recently been paid to the research in domain specific IR, as evidenced by the organization of Genomics and Legal tracks in TREC (Text REtrieval Conference). We think that now is the right time to carry out large scale evaluations on chemistry datasets in order to promote the research in chemical IR in general and chemical patent IR in particular. Accordingly, we propose the organization of a chemical IR track in TREC in order to address the challenges in chemical and patent IR. In this position paper, we present the research questions we will address in the proposed track, our initial plan of the proposed track, and the kind of search tasks we propose for the track. We focus on the design of a new chemical entity search task consisting of two sub-tasks, i.e., chemical entity search and chemical entity relation search.

REFERENCES

• 1: Baron, J.R., Lewis, D.D., and Oard, D.W. (2007) TREC-2006 Legal Track Overview. In Proc. of TREC 2006.
• 2: Barton, J. H. (2003) Non-Obviousness, 43 IDEA: The Journal of Law and Technology 475--508.
• 3: Craswell, N., de Vries, A.P., Soboroff, I. (2006) Overview of the TREC-2005 Enterprise Track. In Proc. of TREC 2005.
• 4: de Vries, A.P., Thom, J.A., Vercoustre, A-M, Craswell N., and Lalmas, M. (2007) INEX 2007 Entity Ranking Track Guidelines - V1. In Pre-Proc. of INitiative for the Evaluation of XML Retrieval.
• 5: Hersh, W., and Bhupatiraju, R.T. (2004) TREC Genomics Track Overview. In Proc. of TREC 2003.
• 6: Lalmas, M., and Baeza-Yates, R. (2009) Structured Document Retrieval, Encyclopedia of Database Systems, O.M. Tamer and L. Ling (Eds.), Springer, May.
• 7: Leach, A.R. and Gillet V.J. An Introduction to Chemoinformatics. Kluwer, Dordrecht, 2003.
• 8: Goran Nenadić , Sophia Ananiadou, Mining semantically related terms from biomedical literature, ACM Transactions on Asian Language Information Processing (TALIP), v.5 n.1, p.22-43, March 2006 [doi>10.1145/1131348.1131351]
• 9: Rosenberg, P. (2000) Patent Law Fundamentals. St. Paul: West Group.
• 10: Soboroff, I., de Vries, A.P. and Craswell, N. (2007) Overview of the TREC 2006 Enterprise Track. In Proc. of TREC 2006.
• 11: Bingjun Sun , Qingzhao Tan , Prasenjit Mitra , C. Lee Giles, Extraction and search of chemical formulae in text documents on the web, Proceedings of the 16th international conference on World Wide Web, May 08-12, 2007, Banff, Alberta, Canada [doi>10.1145/1242572.1242607]
• 12: Tomlinson, S., Oard, D.W., and Baron, J.R. (2008) Overview of the TREC 2007 Legal Track. In Proc. of TREC 2007.
• 13: Walterschneid, E. C. (1998) To Promote the Progress of Useful Arts: American Patent Law and Administration, 1787--1836. William Hein & Co.
• 14: Peter Willett, Chemoinformatics: an application domain for information retrieval techniques, Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, p.393-393, July 25-29, 2004, Sheffield, United Kingdom [doi>10.1145/1008992.1008994]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing

  J. Computer Applications
  J.2 PHYSICAL SCIENCES AND ENGINEERING
      Subjects: Chemistry

General Terms:
Design, Experimentation, Measurement, Performance

Keywords:
chemical information retrieval, chemistry, patent retrieval, prior art search