OCTOPUS: aggressive search of multi-modality data using multifaceted knowledge base

ABSTRACT

An important trend in Web information processing is the support of multimedia retrieval. However, the most prevailing paradigm for multimedia retrieval, content-based retrieval (CBR), is a rather conservative one whose performance depends on a set of specifically defined low-level features and a carefully chosen sample object. In this paper, an aggressive search mechanism called Octopus is proposed which addresses the retrieval of multi-modality data using multifaceted knowledge. In particular, Octopus promotes a novel scenario in which the user supplies seed objects of arbitrary modality as the hint of his information need, and receives a set of multi-modality objects satisfying his need. The foundation of Octopus is a multifaceted knowledge base constructed on a layered graph model (LGM), which describes the relevance between media objects from various perspectives. Link analysis based retrieval algorithm is proposed based on the LGM. A unique relevance feedback technique is developed to update the knowledge base by learning from user behaviors, and to enhance the retrieval performance in a progressive manner. A prototype implementing the proposed approach has been developed to demonstrate its feasibility and capability through illustrative examples.

REFERENCES

• 1: Benitez, A. B., Smith, J. R. and Chang, S. F. "MediaNet: A Multimedia Information Network for Knowledge Representation". In Proc. of the SPIE 2000 Conference on Internet Multimedia Management Systems, vol.4210, 2000.
• 2: Krishna Bharat , Monika R. Henzinger, Improved algorithms for topic distillation in a hyperlinked environment, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.104-111, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.290972]
• 3: Sergey Brin , Lawrence Page, The anatomy of a large-scale hypertextual Web search engine, Proceedings of the seventh international conference on World Wide Web 7, p.107-117, April 1998, Brisbane, Australia
• 4: Shih-Fu Chang , William Chen , Horace J. Meng , Hari Sundaram , Di Zhong, VideoQ: an automated content based video search system using visual cues, Proceedings of the fifth ACM international conference on Multimedia, p.313-324, November 09-13, 1997, Seattle, Washington, United States [doi>10.1145/266180.266382]
• 5: Chen, W. and Chang, S. F. "VISMAP: An Interactive Image/Video Retrieval System Using Visualization and Concept Maps", In Proc. of Int. Conf. on Image Processing (ICIP), Greece, October 2001.
• 6: Jeffrey Dean , Monika R. Henzinger, Finding related pages in the World Wide Web, Proceeding of the eighth international conference on World Wide Web, p.1467-1479, May 1999, Toronto, Canada
• 7: Myron Flickner , Harpreet Sawhney , Wayne Niblack , Jonathan Ashley , Qian Huang , Byron Dom , Monika Gorkani , Jim Hafner , Denis Lee , Dragutin Petkovic , David Steele , Peter Yanker, Query by Image and Video Content: The QBIC System, Computer, v.28 n.9, p.23-32, September 1995 [doi>10.1109/2.410146]
• 8: David Gibson , Jon Kleinberg , Prabhakar Raghavan, Inferring Web communities from link topology, Proceedings of the ninth ACM conference on Hypertext and hypermedia : links, objects, time and space---structure in hypermedia systems: links, objects, time and space---structure in hypermedia systems, p.225-234, June 20-24, 1998, Pittsburgh, Pennsylvania, United States [doi>10.1145/276627.276652]
• 9: Google Search Engine. http://www.google.com.
• 10: Monika R. Henzinger , Allan Heydon , Michael Mitzenmacher , Marc Najork, Measuring index quality using random walks on the Web, Proceeding of the eighth international conference on World Wide Web, p.1291-1303, May 1999, Toronto, Canada
• 11: Thomas Hofmann , Joachim M. Buhmann, Pairwise Data Clustering by Deterministic Annealing, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.19 n.1, p.1-14, January 1997 [doi>10.1109/34.566806]
• 12: Huang, T. S., Mehrotra, S., and Ramchandran, K., "Multimedia analysis and retrieval system (MARS) project," In Proc of 33rd Annual Clinic on Library Application of Data Processing-Digital Image Access and Retrieval, 1996.
• 13: Jon M. Kleinberg, Authoritative sources in a hyperlinked environment, Proceedings of the ninth annual ACM-SIAM symposium on Discrete algorithms, p.668-677, January 25-27, 1998, San Francisco, California, United States
• 14: Ravi Kumar , Prabhakar Raghavan , Sridhar Rajagopalan , Andrew Tomkins, Trawling the Web for emerging cyber-communities, Proceeding of the eighth international conference on World Wide Web, p.1481-1493, May 1999, Toronto, Canada
• 15: Ronny Lempel , Aya Soffer, PicASHOW: pictorial authority search by hyperlinks on the Web, Proceedings of the 10th international conference on World Wide Web, p.438-448, May 01-05, 2001, Hong Kong, Hong Kong [doi>10.1145/371920.372098]
• 16: Qing Li, Web-Based Multimedia Retrieval: Balancing Out between Common Knowledge and Personalized Views, Proceedings of the Second International Conference on Web Information Systems Engineering (WISE'01) Volume 1, p.92, December 03-06, 2001
• 17: Ye Lu , Chunhui Hu , Xingquan Zhu , HongJiang Zhang , Qiang Yang, A unified framework for semantics and feature based relevance feedback in image retrieval systems, Proceedings of the eighth ACM international conference on Multimedia, p.31-37, October 2000, Marina del Rey, California, United States [doi>10.1145/354384.354403]
• 18: Peter Pirolli , James Pitkow , Ramana Rao, Silk from a sow's ear: extracting usable structures from the Web, Proceedings of the SIGCHI conference on Human factors in computing systems: common ground, p.118-125, April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/238386.238450]
• 19: Davood Rafiei , Alberto O. Mendelzon, What is this page known for? Computing Web page reputations, Proceedings of the 9th international World Wide Web conference on Computer networks : the international journal of computer and telecommunications netowrking, p.823-835, June 2000, Amsterdam, The Netherlands
• 20: John R. Smith , Shih-Fu Chang, VisualSEEk: a fully automated content-based image query system, Proceedings of the fourth ACM international conference on Multimedia, p.87-98, November 18-22, 1996, Boston, Massachusetts, United States [doi>10.1145/244130.244151]
• 21: John R. Smith , Shih-Fu Chang, Visually Searching the Web for Content, IEEE MultiMedia, v.4 n.3, p.12-20, July 1997 [doi>10.1109/93.621578]
• 22: Tansley, R., "The Multimedia Thesaurus: An Aid for Multimedia Information Retrieval and Navigation", Master Thesis, Computer Science, University of Southampton, UK, 1998.
• 23: Jun Yang , Yueting Zhuang , Qing Li, Search for Multi-modality Data in Digital Libraries, Proceedings of the Second IEEE Pacific Rim Conference on Multimedia: Advances in Multimedia Information Processing, p.482-489, October 24-26, 2001

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Search process

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Selection process
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Evaluation/methodology

General Terms:
Algorithms, Experimentation, Performance

Keywords:
layered graph model, link analysis, multi-modality data, multifaceted knowledge base, multimedia retrieval, relevance feedback