Placing flickr photos on a map

ABSTRACT

In this paper we investigate generic methods for placing photos uploaded to Flickr on the World map. As primary input for our methods we use the textual annotations provided by the users to predict the single most probable location where the image was taken. Central to our approach is a language model based entirely on the annotations provided by users. We define extensions to improve over the language model using tag-based smoothing and cell-based smoothing, and leveraging spatial ambiguity. Further we demonstrate how to incorporate GeoNames\footnote{http://www.geonames.org visited May 2009}, a large external database of locations. For varying levels of granularity, we are able to place images on a map with at least twice the precision of the state-of-the-art reported in the literature.

REFERENCES

• 1: Shane Ahern , Mor Naaman , Rahul Nair , Jeannie Hui-I Yang, World explorer: visualizing aggregate data from unstructured text in geo-referenced collections, Proceedings of the 7th ACM/IEEE-CS joint conference on Digital libraries, June 18-23, 2007, Vancouver, BC, Canada [doi>10.1145/1255175.1255177]
• 2: Morgan Ames , Mor Naaman, Why we tag: motivations for annotation in mobile and online media, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240772]
• 3: Einat Amitay , Nadav Har'El , Ron Sivan , Aya Soffer, Web-a-where: geotagging web content, Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, July 25-29, 2004, Sheffield, United Kingdom [doi>10.1145/1008992.1009040]
• 4: Lars Backstrom , Jon Kleinberg , Ravi Kumar , Jasmine Novak, Spatial variation in search engine queries, Proceeding of the 17th international conference on World Wide Web, April 21-25, 2008, Beijing, China [doi>10.1145/1367497.1367546]
• 5: Guihong Cao , Jian-Yun Nie , Jianfeng Gao , Stephen Robertson, Selecting good expansion terms for pseudo-relevance feedback, Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, July 20-24, 2008, Singapore, Singapore [doi>10.1145/1390334.1390377]
• 6: David J. Crandall , Lars Backstrom , Daniel Huttenlocher , Jon Kleinberg, Mapping the world's photos, Proceedings of the 18th international conference on World wide web, April 20-24, 2009, Madrid, Spain [doi>10.1145/1526709.1526812]
• 7: Junyan Ding , Luis Gravano , Narayanan Shivakumar, Computing Geographical Scopes of Web Resources, Proceedings of the 26th International Conference on Very Large Data Bases, p.545-556, September 10-14, 2000
• 8: J. Hays and A. A. Efros. im2gps: estimating geographic information from a single image. In Proceedings of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2008.
• 9: Lyndon S. Kennedy , Mor Naaman, Generating diverse and representative image search results for landmarks, Proceeding of the 17th international conference on World Wide Web, April 21-25, 2008, Beijing, China [doi>10.1145/1367497.1367539]
• 10: J. Leidner. Toponym Resolution in Text: Annotation, Evaluation and Applications of Spatial Grounding of Place Names. PhD thesis, University of Edinburgh, 2007.
• 11: Andrew McCallum , Ronald Rosenfeld , Tom M. Mitchell , Andrew Y. Ng, Improving Text Classification by Shrinkage in a Hierarchy of Classes, Proceedings of the Fifteenth International Conference on Machine Learning, p.359-367, July 24-27, 1998
• 12: Qiaozhu Mei , Chao Liu , Hang Su , ChengXiang Zhai, A probabilistic approach to spatiotemporal theme pattern mining on weblogs, Proceedings of the 15th international conference on World Wide Web, May 23-26, 2006, Edinburgh, Scotland [doi>10.1145/1135777.1135857]
• 13: Mor Naaman , Susumu Harada , QianYing Wang , Hector Garcia-Molina , Andreas Paepcke, Context data in geo-referenced digital photo collections, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027573]
• 14: M. Naaman, A. Paepcke, and H. Garcia-Molina. From where to what: metadata sharing for digital photographs with geographic coordinates. On The Move to Meaningful Internet Systems 2003: CoopIS, DOA, and ODBASE, pages 196--217, 2003.
• 15: Simon Overell , Börkur Sigurbjörnsson , Roelof van Zwol, Classifying tags using open content resources, Proceedings of the Second ACM International Conference on Web Search and Data Mining, February 09-12, 2009, Barcelona, Spain [doi>10.1145/1498759.1498810]
• 16: Jonathan Raper , Georg Gartner , Hassan Karimi , Chris Rizos, Applications of location-based services: a selected review, Journal of Location Based Services, v.1 n.2, p.89-111, June 2007 [doi>10.1080/17489720701862184]
• 17: Tye Rattenbury , Nathaniel Good , Mor Naaman, Towards automatic extraction of event and place semantics from flickr tags, Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, July 23-27, 2007, Amsterdam, The Netherlands [doi>10.1145/1277741.1277762]
• 18: M. Richardson and P. Domingos. The intelligent surfer: Probabilistic combination of link and content information in pagerank. In NIPS '01: Advances in Neural Information Processing Systems, 2001.
• 19: Pavel Serdyukov , Henning Rode , Djoerd Hiemstra, Modeling multi-step relevance propagation for expert finding, Proceeding of the 17th ACM conference on Information and knowledge management, October 26-30, 2008, Napa Valley, California, USA [doi>10.1145/1458082.1458232]
• 20: Azadeh Shakery , ChengXiang Zhai, A probabilistic relevance propagation model for hypertext retrieval, Proceedings of the 15th ACM international conference on Information and knowledge management, November 06-11, 2006, Arlington, Virginia, USA [doi>10.1145/1183614.1183693]
• 21: Börkur Sigurbjörnsson , Roelof van Zwol, Flickr tag recommendation based on collective knowledge, Proceeding of the 17th international conference on World Wide Web, April 21-25, 2008, Beijing, China [doi>10.1145/1367497.1367542]
• 22: C. Torniai, S. Battle, and S. Cayzer. Sharing, discovering and browsing geotagged pictures on the web. Technical report, Digital Media Systems Laboratory, HP Laboratories Bristol, 2007.
• 23: Kentaro Toyama , Ron Logan , Asta Roseway, Geographic location tags on digital images, Proceedings of the eleventh ACM international conference on Multimedia, November 02-08, 2003, Berkeley, CA, USA [doi>10.1145/957013.957046]
• 24: Srinivas Vadrevu , Ya Zhang , Belle Tseng , Gordon Sun , Xin Li, Identifying regional sensitive queries in web search, Proceeding of the 17th international conference on World Wide Web, April 21-25, 2008, Beijing, China [doi>10.1145/1367497.1367718]
• 25: Chong Wang , Jinggang Wang , Xing Xie , Wei-Ying Ma, Mining geographic knowledge using location aware topic model, Proceedings of the 4th ACM workshop on Geographical information retrieval, November 09-09, 2007, Lisbon, Portugal [doi>10.1145/1316948.1316967]
• 26: Kilian Quirin Weinberger , Malcolm Slaney , Roelof Van Zwol, Resolving tag ambiguity, Proceeding of the 16th ACM international conference on Multimedia, October 26-31, 2008, Vancouver, British Columbia, Canada [doi>10.1145/1459359.1459375]
• 27: ChengXiang Zhai , John Lafferty, Two-stage language models for information retrieval, Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, August 11-15, 2002, Tampere, Finland [doi>10.1145/564376.564387]
• 28: Ziming Zhuang , Cliff Brunk , C. Lee Giles, Modeling and visualizing geo-sensitive queries based on user clicks, Proceedings of the first international workshop on Location and the web, p.73-76, April 22-22, 2008, Beijing, China [doi>10.1145/1367798.1367811]
• 29: Wenbo Zong , Dan Wu , Aixin Sun , Ee-Peng Lim , Dion Hoe-Lian Goh, On assigning place names to geography related web pages, Proceedings of the 5th ACM/IEEE-CS joint conference on Digital libraries, June 07-11, 2005, Denver, CO, USA [doi>10.1145/1065385.1065464]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Search process

General Terms:
Algorithms, Experimentation, Measurement, Performance

Keywords:
flickr, image localization, language models