Context data in geo-referenced digital photo collections

ABSTRACT

Given time and location information about digital photographs we can automatically generate an abundance of related contextual metadata, using off-the-shelf and Web-based data sources. Among these are the local daylight status and weather conditions at the time and place a photo was taken. This metadata has the potential of serving as memory cues and filters when browsing photo collections, especially as these collections grow into the tens of thousands and span dozens of years.

We describe the contextual metadata that we automatically assemble for a photograph, given time and location, as well as a browser interface that utilizes that metadata. We then present the results of a user study and a survey that together expose which categories of contextual metadata are most useful for recalling and finding photographs. We identify among still unavailable metadata categories those that are most promising to develop next.

REFERENCES

• 1: Jonathan Ashley , Myron Flickner , James Hafner , Denis Lee , Wayne Niblack , Dragutin Petkovic, The query by image content (QBIC) system, Proceedings of the 1995 ACM SIGMOD international conference on Management of data, p.475, May 22-25, 1995, San Jose, California, United States
• 2: Benjamin B. Bederson, PhotoMesa: a zoomable image browser using quantum treemaps and bubblemaps, Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida [doi>10.1145/502348.502359]
• 3: V. Bush. As we may think. The Atlantic Monthly, July 1945.
• 4: Matthew Cooper , Jonathan Foote , Andreas Girgensohn , Lynn Wilcox, Temporal event clustering for digital photo collections, Proceedings of the eleventh ACM international conference on Multimedia, November 02-08, 2003, Berkeley, CA, USA [doi>10.1145/957013.957093]
• 5: Jim Gemmell , Gordon Bell , Roger Lueder , Steven Drucker , Curtis Wong, MyLifeBits: fulfilling the Memex vision, Proceedings of the tenth ACM international conference on Multimedia, December 01-06, 2002, Juan-les-Pins, France [doi>10.1145/641007.641053]
• 6: Adrian Graham , Hector Garcia-Molina , Andreas Paepcke , Terry Winograd, Time as essence for photo browsing through personal digital libraries, Proceedings of the 2nd ACM/IEEE-CS joint conference on Digital libraries, July 14-18, 2002, Portland, Oregon, USA [doi>10.1145/544220.544301]
• 7: Susumu Harada , Mor Naaman , Yee Jiun Song , QianYing Wang , Andreas Paepcke, Lost in memories: interacting with photo collections on PDAs, Proceedings of the 4th ACM/IEEE-CS joint conference on Digital libraries, June 07-11, 2004, Tuscon, AZ, USA [doi>10.1145/996350.996425]
• 8: L. L. Hill, J. Frew, and Q. Zheng. Geographic names - the implementation of a gazetteer in a georeferenced digital library. CNRI D-Lib Magazine, January 1999.
• 9: E. Hjelmas and B. K. Low. Face detection: a survey. Computer Vision and Image Understanding, 83(3):236 -- 74, SEP 2001.
• 10: J. Luo and A. Savakis. Indoor vs. outdoor classification of consumer photographs. In Proceedings of the international conference on Image Processing (ICIP), 2001.
• 11: M. Naaman, S. Harada, Q. Wang, and A. Paepcke. Adventures in space and time: Browsing personal collections of geo-referenced digital photographs. Technical report, Stanford University, April 2004. Submitted for Publication.
• 12: M. Naaman, A. Paepcke, and H. Garcia-Molina. From where to what: Metadata sharing for digital photographs with geographic coordinates. In 10th International Conference on Cooperative Information Systems (CoopIS), 2003.
• 13: Mor Naaman , Yee Jiun Song , Andreas Paepcke , Hector Garcia-Molina, Automatic organization for digital photographs with geographic coordinates, Proceedings of the 4th ACM/IEEE-CS joint conference on Digital libraries, June 07-11, 2004, Tuscon, AZ, USA [doi>10.1145/996350.996366]
• 14: Kerry Rodden , Kenneth R. Wood, How do people manage their digital photographs?, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642682]
• 15: Risto Sarvas , Erick Herrarte , Anita Wilhelm , Marc Davis, Metadata creation system for mobile images, Proceedings of the 2nd international conference on Mobile systems, applications, and services, June 06-09, 2004, Boston, MA, USA [doi>10.1145/990064.990072]
• 16: Direct Annotation: A Drag-and-Drop Strategy for Labeling Photos, Proceedings of the International Conference on Information Visualisation, p.88, July 19-21, 2000
• 17: Kentaro Toyama , Ron Logan , Asta Roseway, Geographic location tags on digital images, Proceedings of the eleventh ACM international conference on Multimedia, November 02-08, 2003, Berkeley, CA, USA [doi>10.1145/957013.957046]
• 18: R. C. Veltkamp and M. Tanase. Content-based image retrieval systems: A survey. Technical Report TR UU-CS-2000-34 (revised version), Department of Computing Science, Utrecht University, October 2002.
• 19: W. Wagenaar. My memory: A study of autobiographical memory over six years. Cognitive psychology, 18:225--252, 1986.
• 20: Ka-Ping Yee , Kirsten Swearingen , Kevin Li , Marti Hearst, Faceted metadata for image search and browsing, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642681]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems

General Terms:
Human Factors

Keywords:
context, geo-referenced digital photos, photo collections