Mapping the world's photos

ABSTRACT

We investigate how to organize a large collection of geotagged photos, working with a dataset of about 35 million images collected from Flickr. Our approach combines content analysis based on text tags and image data with structural analysis based on geospatial data. We use the spatial distribution of where people take photos to define a relational structure between the photos that are taken at popular places. We then study the interplay between this structure and the content, using classification methods for predicting such locations from visual, textual and temporal features of the photos. We find that visual and temporal features improve the ability to estimate the location of a photo, compared to using just textual features. We illustrate using these techniques to organize a large photo collection, while also revealing various interesting properties about popular cities and landmarks at a global scale.

REFERENCES

• 1: Shane Ahern , Mor Naaman , Rahul Nair , Jeannie Hui-I Yang, World explorer: visualizing aggregate data from unstructured text in geo-referenced collections, Proceedings of the 7th ACM/IEEE-CS joint conference on Digital libraries, June 18-23, 2007, Vancouver, BC, Canada [doi>10.1145/1255175.1255177]
• 2: Dorin Comaniciu , Peter Meer, Mean Shift: A Robust Approach Toward Feature Space Analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.5, p.603-619, May 2002 [doi>10.1109/34.1000236]
• 3: G. Csurka, C. Dance, L. Fan, J. Willamowski, C. Bray. Visual categorization with bags of keypoints. Statistical Learning in Computer Vision, ECCV, 2004.
• 4: http://www.flickr.com/map/
• 5: W. Freeman. The generic viewpoint assumption in a framework for visual perception, Nature, 368(6471), 1994.
• 6: Scott A. Golder , Bernardo A. Huberman, Usage patterns of collaborative tagging systems, Journal of Information Science, v.32 n.2, p.198-208, April 2006 [doi>10.1177/0165551506062337]
• 7: Scott Golder, Measuring social networks with digital photograph collections, Proceedings of the nineteenth ACM conference on Hypertext and hypermedia, June 19-21, 2008, Pittsburgh, PA, USA [doi>10.1145/1379092.1379104]
• 8: James Hays , Alexei A. Efros, Scene completion using millions of photographs, ACM SIGGRAPH 2007 papers, August 05-09, 2007, San Diego, California
• 9: J. Hays, A. Efros. IM2GPS: estimating geographic information from a single image, CVPR 2008.
• 10: Thorsten Joachims, Making large-scale support vector machine learning practical, Advances in kernel methods: support vector learning, MIT Press, Cambridge, MA, 1999
• 11: Lyndon S. Kennedy , Mor Naaman, Generating diverse and representative image search results for landmarks, Proceeding of the 17th international conference on World Wide Web, April 21-25, 2008, Beijing, China [doi>10.1145/1367497.1367539]
• 12: Ravi Kumar , Jasmine Novak , Andrew Tomkins, Structure and evolution of online social networks, Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, August 20-23, 2006, Philadelphia, PA, USA [doi>10.1145/1150402.1150476]
• 13: Ravi Kumar , Prabhakar Raghavan , Sridhar Rajagopalan , Andrew Tomkins, Trawling the Web for emerging cyber-communities, Computer Networks: The International Journal of Computer and Telecommunications Networking, v.31 n.11-16, p.1481-1493, May 17, 1999
• 14: K. Lerman, L. Jones. Social Browsing on Flickr Intl. Conf. on Weblogs and Social Media, 2007.
• 15: Jure Leskovec , Lars Backstrom , Ravi Kumar , Andrew Tomkins, Microscopic evolution of social networks, Proceeding of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, August 24-27, 2008, Las Vegas, Nevada, USA [doi>10.1145/1401890.1401948]
• 16: David G. Lowe, Distinctive Image Features from Scale-Invariant Keypoints, International Journal of Computer Vision, v.60 n.2, p.91-110, November 2004 [doi>10.1023/B:VISI.0000029664.99615.94]
• 17: Christopher D. Manning , Prabhakar Raghavan , Hinrich Schtze, Introduction to Information Retrieval, Cambridge University Press, New York, NY, 2008
• 18: C. Marlow, M. Naaman, D. Boyd, M. Davis. Position Paper, Tagging, Taxonomy, Flickr, Article, ToRead, Collaborative Web Tagging Workshop (at WWW), 2006.
• 19: Aude Oliva , Antonio Torralba, Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope, International Journal of Computer Vision, v.42 n.3, p.145-175, May-June 2001 [doi>10.1023/A:1011139631724]
• 20: Shilad Sen , Shyong K. Lam , Al Mamunur Rashid , Dan Cosley , Dan Frankowski , Jeremy Osterhouse , F. Maxwell Harper , John Riedl, tagging, communities, vocabulary, evolution, Proceedings of the 2006 20th anniversary conference on Computer supported cooperative work, November 04-08, 2006, Banff, Alberta, Canada [doi>10.1145/1180875.1180904]
• 21: Jianbo Shi , Jitendra Malik, Normalized Cuts and Image Segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.22 n.8, p.888-905, August 2000 [doi>10.1109/34.868688]
• 22: I. Simon, N. Snavely, S. Seitz. Scene summarization for online image collections, ICCV 2007.
• 23: Noah Snavely , Steven M. Seitz , Richard Szeliski, Photo tourism: exploring photo collections in 3D, ACM SIGGRAPH 2006 Papers, July 30-August 03, 2006, Boston, Massachusetts
• 24: H.T. Welser, E. Gleave, D. Fisher, M. Smith. Visualizing the Signatures of Social Roles in Online Discussion Groups, J. Social Structure, 2007.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications
          Subjects: Data mining

Additional Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications
          Subjects: Image databases; Spatial databases and GIS

General Terms:
Measurement, Theory

Keywords:
geolocation, photo collections