Image annotation using personal calendars as context

ABSTRACT

In this paper, we introduce the idea of using the context of a personal calendar for labeling photo collections. Calendar event annotations are matched to images based on image capture time, and a Naïve Bayes model considers features from the calendar events as well as from computer vision-based image analysis to determine if the image actually matches the calendar event. This approach has the benefit that it requires no extra annotation from the consumer, since most people already keep calendars. In our test collections, 36% of personal images could be tagged with a label from a personal calendar. Note that our preliminary results represent a lower bound on the performance that is possible because all the system components are expected to improve over time. As people migrate toward digital calendars, we can also expect more consistency in their calendar labels, which should improve the annotation accuracy.

REFERENCES

• 1: Cao, L., Luo, J., Kautz H. and Huang, T. Annotating Collections of Photos Using Hierarchical Event and Scene Models, In Proc CVPR, 2008.
• 2: Navneet Dalal , Bill Triggs, Histograms of Oriented Gradients for Human Detection, Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 1, p.886-893, June 20-26, 2005 [doi>10.1109/CVPR.2005.177]
• 3: Jones, M. and Viola, P., Fast Multiview Face Detector, In Proc. CVPR, 2003.
• 4: Loui, A. and A. Savakis. Automated Event Clustering and Quality Screening of Consumer Pictures for Digital Albuming, IEEE Trans. Multimedia, 5(3): 390--402, 2003.
• 5: Mor Naaman , Hector Garcia-Molina, Leveraging geo-referenced digital photographs, Stanford University, Stanford, CA, 2005
• 6: Carman Gerard Neustaedter, Domestic awareness and the role of family calendars, University of Calgary, Calgary, Alta., Canada, 2007
• 7: U.S. Social Security Administration. Baby Name Database. http://www.socialsecurity.gov/OACT/babynames/.
• 8: http://en.wikipedia.org/wiki/Likert_scale

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.4 Applications
          Subjects: Computer vision

General Terms:
Algorithms, Experimentation, Human Factors

Keywords:
calendar, content-based image retrieval, image annotation