Mining city landmarks from blogs by graph modeling

ABSTRACT

Recent years have witnessed great prosperity in community-contributed multimedia. Discovering, extracting, and summarizing knowledge from these data enables us to make better sense of the world. In this paper, we report our work on mining famous city landmarks from blogs for personalized tourist suggestions. Our main contribution is a graph modeling framework to discover city landmarks by mining blog photo correlations with community supervision. This modeling fuses context, content, and community information in a style that simulates both static (PageRank) and dynamic (HITS) ranking models to highlight representative data from the consensus of blog users.

Preliminary, we identify geographical locations of page contents to harvest city sight photos from Web blogs, based on which we structure these photos into a Scene-View hierarchy* within each city. Our graph modeling consists of two phases: First, within a given scene, we present a PhotoRank algorithm to discover its representative views, which analogizes PageRank to model context and content photo correlations for graph-based popularity propagation. Second, among scenes within each city, we present a Landmark-HITS model to discover city landmarks, which integrates author correlations to infer scene popularity in a semi-supervised reinforcement manner. Based on graph modeling, we further achieve personalized tourist suggestions by the collaborative filtering of tourism logs and author correlations. Based on a real-world dataset from Windows Live Spaces blogs containing nearly 400,000 sight photos, we have deployed our framework in a VisualTourism system, with comparisons to state-of-the-arts. We also investigate how the city popularities, user locations (e.g. Asian or Euro. blog users), and sequential events (e.g. Olympic Games) influence our Landmark discovery results and the tourist suggestion tendencies.

REFERENCES

• 1: S. Brin and L. Page, "The anatomy of a large-scale hypertextual Web search engine," WWW, 1998.
• 2: G. David; D. Nichols, B. M. Oki, D. Terry, "Using collaborative filtering to weave an information tapestry". Communications of the ACM, 35 (12): 61--70, 1992.
• 3: H. Mannilla, H. Toivonen, and A. Verkamo, "Discovery of frequent episodes in event sequences," ACM SIGKDD, 1997.
• 4: C. L. Wayne, "Multilingual topic detection and tracking: successful research enabled by corpora and evaluation," International Conference of the Language Resources and Evaluation, 2000.
• 5: C. Tsai, A. Qamra, and E. Chang, "Extent: inferring image metadata from context and content," ICME, 2005.
• 6: Y. Hu, M.-J. Li, Z. Li, W.-Y. Ma, "Discovering authoritative news sources and top news stories," Information Retrieval Technology, Section 2A: Web Information Retrieval, 2006.
• 7: J. Kleinberg, "Authoritative sources in a hyperlinked environment," Journal of the ACM, Vol. 46, No. 5, pp. 604--622, 1999.
• 8: Y. Jing, S. Baluju, "PageRank for product image search," WWW, 2008.
• 9: I. Simmon, N. Snavely, and S. M. Seitz, "Scene summarization for online image collections," ICCV, 2007.
• 10: B. Sigurbjornsson and R. Van Zwol, "Flickr tag recommendation based on collective knowledge," WWW, 2008.
• 11: N. O'Hare, C. Gurrin, G. Jones and A. Smeaton, "Combination of content analysis and context features for digital photograph retrieval," European Workshop on the Integration of Knowledge, Semantic and Digital Media Technologies, 2005.
• 12: F. Jing, L. Zhang, W.-Y. Ma, "VirtualTour: an online travel assistant based on high quality images," ACM Multimedia, 2006.
• 13: L. Kennedy, M. Naaman, S. Ahern, R. Nail, T. Rattenbury, "How Flickr helps us make sense of the world: context and content in community-contributed media collections," ACM Multimedia, 2007.
• 14: L. Kennedy, M. Naaman, "Generating Diverse and Representative Images Search Results for Landmarks," ACM KDD, 2007.
• 15: C. Wang, X. Xie, L. Wang, Y. Lu, W-Y. Ma, "Detecting Geographic Locations from Web Resources", GIS, 2005
• 16: D. G. Lowe, "Distinctive image features from scale-invariant keypoints," IJCV, 2004.
• 17: D. Nister and H. Stewenius, "Scalable recognition with a vocabulary tree," CVPR, 2006.
• 18: M.-L. Jia, X. Fan, X. Xie, M.-J. Li, W.-Y. Ma, "Photo-to-search: Using camera phones to inquire of the surrounding world," Mobile Data Management, 2006.
• 19: G. Salton, and C. Buckley, "Term-weighting approaches in automatic text retrieval," Inform. Process. and Management, 1998.
• 20: G. Schindler, M. Brown and R. Szeliski, "City-scale location recognition," CVPR, 2007.
• 21: D. Cai, X. He, J. Wen, W. Y. Ma, "Block-level link analysis," SIGIR, 2004.
• 22: G. Schindler, M. Brown and R. Szeliski, "City-scale location recognition," CVPR, 2007.
• 23: P. Cesar, D. Bulterman et al. "Enhancing social sharing of videos: fragment, annotate, enrich, and share", ACM Multimedia, 2008.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.10 Vision and Scene Understanding

General Terms:
Algorithms, Experimentation, Theory

Keywords:
HITS, PageRank, collaborative filtering, landmark discovery, location extraction, tourist suggestion, user study, web blog