Generating summaries and visualization for large collections of geo-referenced photographs

ABSTRACT

We describe a framework for automatically selecting a summary set of photos from a large collection of geo-referenced photographs. Such large collections are inherently difficult to browse, and become excessively so as they grow in size, making summaries an important tool in rendering these collections accessible. Our summary algorithm is based on spa-tial patterns in photo sets, as well as textual-topical patterns and user (photographer) identity cues. The algorithm can be expanded to support social, temporal, and other factors. The summary can thus be biased by the content of the query, the user making the query, and the context in which the query is made.A modified version of our summarization algorithm serves as a basis for a new map-based visualization of large collections of geo-referenced photos, called Tag Maps. Tag Maps visualize the data by placing highly representative textual tags on relevant map locations in the viewed region, effectively providing a sense of the important concepts embodied in the collection.An initial evaluation of our implementation on a set of geo-referenced photos shows that our algorithm and visualization perform well, producing summaries and views that are highly rated by users.

REFERENCES

• 1: Jaime Carbonell , Jade Goldstein, The use of MMR, diversity-based reranking for reordering documents and producing summaries, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.335-336, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.291025]
• 2: Matthew Cooper , Jonathan Foote , Andreas Girgensohn , Lynn Wilcox, Temporal event clustering for digital photo collections, Proceedings of the eleventh ACM international conference on Multimedia, November 02-08, 2003, Berkeley, CA, USA [doi>10.1145/957013.957093]
• 3: Marc Davis , Simon King , Nathan Good , Risto Sarvas, From context to content: leveraging context to infer media metadata, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027572]
• 4: Marc Davis, Michael Smith, Fred Stentiford, Adetokunbo Bambidele, John Canny, Nathan Good, Simon King, and Rajkumar Janakiraman. Using context and similarity for face and location identification. In Proceedings of the IS&T/SPIE 18th Annual Symposium on Electronic Imaging Science and Technology, 2006.
• 5: Flickr.com. http://www. ickr.com.
• 6: Ullas Gargi. Consumer media capture: Time-based analysis and event clustering. Technical Report HPL-2003-165, HP Laboratories, August 2003.
• 7: Jacob Goldberger and Tamir Tassa. The hungarian clustering method. Technical report, 2006. Submitted for publication.
• 8: Adrian Graham , Hector Garcia-Molina , Andreas Paepcke , Terry Winograd, Time as essence for photo browsing through personal digital libraries, Proceedings of the 2nd ACM/IEEE-CS joint conference on Digital libraries, July 14-18, 2002, Portland, Oregon, USA [doi>10.1145/544220.544301]
• 9: Dhiraj Joshi , James Z. Wang , Jia Li, The story picturing engine: finding elite images to illustrate a story using mutual reinforcement, Proceedings of the 6th ACM SIGMM international workshop on Multimedia information retrieval, October 15-16, 2004, New York, NY, USA [doi>10.1145/1026711.1026732]
• 10: John Lafferty , Chengxiang Zhai, Document language models, query models, and risk minimization for information retrieval, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.111-119, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.383970]
• 11: Mor Naaman, Andreas Paepcke, and Hector Garcia-Molina. From where to what: Metadata sharing for digital photographs with geographic coordinates. In 10th International Conference on Cooperative Information Systems (CoopIS), 2003.
• 12: Mor Naaman , Yee Jiun Song , Andreas Paepcke , Hector Garcia-Molina, Automatic organization for digital photographs with geographic coordinates, Proceedings of the 4th ACM/IEEE-CS joint conference on Digital libraries, June 07-11, 2004, Tuscon, AZ, USA [doi>10.1145/996350.996366]
• 13: Rahul Nair , Nick Reid , Marc Davis, Photo LOI: browsing multi-user photo collections, Proceedings of the 13th annual ACM international conference on Multimedia, November 06-11, 2005, Hilton, Singapore [doi>10.1145/1101149.1101187]
• 14: N. O'Hare, C. Gurrin, G. J.F. Jones, and A. F. Smeaton. Combination of content analysis and context features for digital photograph retrieval. In 2nd IEE European Workshop on the Integration of Knowledge, Semantic and Digital Media Technologies, 2005.
• 15: A. Pigeau and M. Gelgon. Organizing a personal image collection with statistical model-based ICL clustering on spatio-temporal camera phone meta-data. Journal of Visual Communication and Image Representation, 15(3):425--445, September 2004.
• 16: Aran Qamra, C. Tsai, and Edward Y. Chang. A scalable system for landmark recognition in digital photographs. Technical report, UCSB, 2005.
• 17: Patrick Schmitz. Inducing ontology from ickr tags. In Workshop on Collaborative Web Tagging, 2006.
• 18: Susan Sontag. On Photography. Picador, New York, NY, 1977.
• 19: Kentaro Toyama , Ron Logan , Asta Roseway, Geographic location tags on digital images, Proceedings of the eleventh ACM international conference on Multimedia, November 02-08, 2003, Berkeley, CA, USA [doi>10.1145/957013.957046]
• 20: Hal R. Varian, Economics and search, ACM SIGIR Forum, v.33 n.1, p.1-5, Fall 1999 [doi>10.1145/331403.331404]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

General Terms:
Algorithms, Human Factors

Keywords:
clustering, collection visualization, geo-referenced photos, image search, photo collections, summarization