Structured correspondence topic models for mining captioned figures in biological literature

ABSTRACT

A major source of information (often the most crucial and informative part) in scholarly articles from scientific journals, proceedings and books are the figures that directly provide images and other graphical illustrations of key experimental results and other scientific contents. In biological articles, a typical figure often comprises multiple panels, accompanied by either scoped or global captioned text. Moreover, the text in the caption contains important semantic entities such as protein names, gene ontology, tissues labels, etc., relevant to the images in the figure. Due to the avalanche of biological literature in recent years, and increasing popularity of various bio-imaging techniques, automatic retrieval and summarization of biological information from literature figures has emerged as a major unsolved challenge in computational knowledge extraction and management in the life science. We present a new structured probabilistic topic model built on a realistic figure generation scheme to model the structurally annotated biological figures, and we derive an efficient inference algorithm based on collapsed Gibbs sampling for information retrieval and visualization. The resulting program constitutes one of the key IR engines in our SLIF system that has recently entered the final round (4 out 70 competing systems) of the Elsevier Grand Challenge on Knowledge Enhancement in the Life Science. Here we present various evaluations on a number of data mining tasks to illustrate our method.

REFERENCES

• 1: A. Ahmed, E. P. Xing, W. W. Cohen, and R. F. Murphy. Structured correspondence topic models for mining captioned figures in biological literature. Technical report, CMU, 2009.
• 2: Kobus Barnard , Pinar Duygulu , David Forsyth , Nando de Freitas , David M. Blei , Michael I. Jordan, Matching words and pictures, The Journal of Machine Learning Research, 3, 3/1/2003
• 3: David M. Blei , Michael I. Jordan, Modeling annotated data, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860460]
• 4: C. Chemudugunta, P. Smyth, and M. Steyvers. Modeling general and specific aspects of documents with a probabilistic topic model. NIPS, 2006.
• 5: William W. Cohen , Richard Wang , Robert F. Murphy, Understanding captions in biomedical publications, Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining, August 24-27, 2003, Washington, D.C. [doi>10.1145/956750.956809]
• 6: S. Deerwester, S. Dumais, G. Furnas, T. Lanouauer, and R. Harshman. Indexing by latent semantic analysis. Journal of the American Society for Information Science, 1990.
• 7: A. Gelman, J. Carlin, H. Stern, and D. Rubin. Bayesian Data Analysis 2nd edition. Chapman-Hall, 2003.
• 8: T. Griffiths and M. Steyvers. Finding scientific topics. PNAS, 101:5228--5235, 2004.
• 9: V. Jain, E. Learned-Miller, and A. McCallum. People-lda: Anchoring topics to people using face recognition. ICCV, 2007.
• 10: J. Jeon , V. Lavrenko , R. Manmatha, Automatic image annotation and retrieval using cross-media relevance models, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860459]
• 11: Z. Kou, W. W. Cohen, and R. F. Murphy. High-recall protein entity recognition using a dictionary. ISMB, 2005.
• 12: F. Leitner and A. Valencia. A text-mining perspective on the requirements for electronically annotated abstracts. FEBS Letters, 582(8):1178--1181, 2008.
• 13: Robert F. Murphy , Meel Velliste , Gregory Porreca, Robust numerical features for description and classification of subcellular location patterns in fluorescence microscope images, Journal of VLSI Signal Processing Systems, v.35 n.3, p.311-321, November 2003 [doi>10.1023/B:VLSI.0000003028.71666.44]
• 14: Robert F. Murphy , Meel Velliste , Jie Yao , Gregory Porreca, Searching Online Journals for Fluorescence Microscope Images Depicting Protein Subcellular Location Patterns, Proceedings of the 2nd IEEE International Symposium on Bioinformatics and Bioengineering, p.119, March 04-06, 2001
• 15: David Newman , Chaitanya Chemudugunta , Padhraic Smyth, Statistical entity-topic models, Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, August 20-23, 2006, Philadelphia, PA, USA [doi>10.1145/1150402.1150487]
• 16: J. Pan, H. Yang, C. Faloutsos, and P. Duygulu. Gcap: Graph-based automatic image captioning. Workshop on Multimedia Data and Document Engineering, 2004.
• 17: Jay M. Ponte , W. Bruce Croft, A language modeling approach to information retrieval, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.275-281, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.291008]
• 18: J. Yang, Y. Liu, E. P. Xing, and A. Hauptmann. Harmonium-based models for semantic video representation and classification. SDM, 2005.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning

Additional Classification:
  G. Mathematics of Computing
  G.3 PROBABILITY AND STATISTICS

  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications
          Subjects: Image databases; Data mining

General Terms:
Algorithms, Experimentation

Keywords:
bioinformatics, gibbs sampling, information retrieval, multimodal data, topic models