Object-name search by visual appearance and spatio-temporal descriptions

ABSTRACT

When a user searches the Web for information about a target object by submitting a keyword-based query to such a conventional Web search engine as Google, the precision and recall of the search results depend a great deal on whether or not s/he has known exactly the concrete name of the target object. However, the user does not always know the concrete name of any target object that s/he has encountered in the real world and wanted information about. In this paper, we propose an application system of Object-Name Search that helps her/him to identify the concrete name of the target object by such ambiguous features as its class-name, visual appearance and spatio-temporal information. When the user inputs a class-name, visual appearance and/or real-world context descriptions, our system returns not only concrete object-names ranked by her/his specification but also their typical images, visual appearance and spatio-temporal descriptions. And then the user can also modify her/his original specification repeatedly by using their typical features as a useful reference.

REFERENCES

• 1: Sharon A. Caraballo, Automatic construction of a hypernym-labeled noun hierarchy from text, Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics, p.120-126, June 20-26, 1999, College Park, Maryland [doi>10.3115/1034678.1034705]
• 2: P. Duygulu , Kobus Barnard , J. F. G. de Freitas , David A. Forsyth, Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary, Proceedings of the 7th European Conference on Computer Vision-Part IV, p.97-112, May 28-31, 2002
• 3: Flickr. http://www.flickr.com/, 2008.
• 4: Google, http://www.google.co.jp/, 2008.
• 5: S. Hattori, H. Ohshima, S. Oyama, and K. Tanaka. Mining the Web for hyponymy relations based on property inheritance. In Proceedings of the 10th Asia-Pacific Web Conference (APWeb'08), LNCS Vol.4976, pages 99--110, April 2008.
• 6: Shun Hattori , Katsumi Tanaka, Extracting Concept Hierarchy Knowledge from the Web Based on Property Inheritance and Aggregation, Proceedings of the 2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology, p.432-437, December 09-12, 2008 [doi>10.1109/WIIAT.2008.394]
• 7: S. Hattori and K. Tanaka. Search the Web for typical images based on extracting color-names from the Web and converting them to color-features. DBSJ Letters, 6(4):1--4, March 2008.
• 8: S. Hattori, T. Tezuka, and K. Tanaka. Mining the Web for appearance description. In Proceedings of the 18th International Conference on Database and Expert Systems Applications (DEXA'07), LNCS Vol.4653, pages 790--800, September 2007.
• 9: Marti A. Hearst, Automatic acquisition of hyponyms from large text corpora, Proceedings of the 14th conference on Computational linguistics, August 23-28, 1992, Nantes, France [doi>10.3115/992133.992154]
• 10: H. Kim, H. Kim, I. Choi, and M. Kim. Finding relations from a large corpus using generalized patterns. International Journal of Information Technology, 12(7):22--29, 2006.
• 11: G. Miller, R. Beckwith, C. Fellbaum, D. Gross, and K. Miller. Introduction to WordNet: an online lexical database. International Journal of Lexicography, 3(4):235--312, 1990.
• 12: Mark Sanderson , Bruce Croft, Deriving concept hierarchies from text, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.206-213, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312679]
• 13: John R. Smith , Shih-Fu Chang, VisualSEEk: a fully automated content-based image query system, Proceedings of the fourth ACM international conference on Multimedia, p.87-98, November 18-22, 1996, Boston, Massachusetts, United States [doi>10.1145/244130.244151]
• 14: Max Völkel , Markus Krötzsch , Denny Vrandecic , Heiko Haller , Rudi Studer, Semantic Wikipedia, Proceedings of the 15th international conference on World Wide Web, May 23-26, 2006, Edinburgh, Scotland [doi>10.1145/1135777.1135863]
• 15: Wikipedia. http://ja.wikipedia.org/, 2008.
• 16: WordNet. http://wordnet.princeton.edu/, 2008.
• 17: Yahoo! Kids Picture-Book. http://contents.kids.yahoo.co.jp/zukan/, 2008.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.4 Systems and Software

General Terms:
Algorithms

Keywords:
appearance, context-awareness, impression, object-information extraction, object-name search, spatiotemporal