Exploring the potentials of combining photo annotating tasks with instant messaging fun

ABSTRACT

The combination of photo annotation tasks and with instant messaging fun offers great potentials to both end-users and researchers. In this paper, we first describe our prototype system that allows users to share and annotate digital photos over the Internet while they are chatting online. In addition to manual annotation, our system can extract information from conversations to generate extra annotations. The advantage of using our system is that the boring and tedious task of annotating photos is turned into an essential part of an attractive fun activity, viz. online chatting. Extracting meaningful information from instant messages is challenged by abbreviations, pronouns, jargon, ellipsis, grammatical errors, ambiguities and asynchrony, all of which frequently appear in message conversations. In the paper we provide a roadmap, i.e. a systematic analysis of linguistic aspects of automated interpretation of message conversations.

REFERENCES

• 1: Benjamin B. Bederson, PhotoMesa: a zoomable image browser using quantum treemaps and bubblemaps, Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida [doi>10.1145/502348.502359]
• 2: Mills, T. J., Pye, D., Sinclair, D., Wood, K. R.: Shoebox: A digital photo management system. Technical Report 2000.10, AT&T Laboratories Cambridge, 2000.
• 3: Marko Balabanović , Lonny L. Chu , Gregory J. Wolff, Storytelling with digital photographs, Proceedings of the SIGCHI conference on Human factors in computing systems, p.564-571, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332505]
• 4: Kerry Rodden , Wojciech Basalaj , David Sinclair , Kenneth Wood, Does organisation by similarity assist image browsing?, Proceedings of the SIGCHI conference on Human factors in computing systems, p.190-197, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365097]
• 5: Platt, J. C., Czerwinski, M., Field, B. A.: PhotoTOC: Automatic clustering for browsing personal photographs. MSR-TR-2002-17, Microsoft Research, 2002.
• 6: Qian, Y., Feijs, L. M. G: Stepwise Concept Navigation. In de Moor, A., Ganter, B. (Eds.): Using Conceptual Structures of ICCS 2003. Germany. 2003. 255--268
• 7: Pentland, A., Picard, R., Sclaroff, S.: Photobook: Content-based Manipulation of Image Databases. In SPIE Proceedings Vol. 2185 (1994) 34--47
• 8: Jain, A., Vailaya, A.: Image retrieval using color and shape. Journal of Pattern Recognition 29(8): 1233--1244, August 1996.
• 9: John R. Smith , Shih-Fu Chang, VisualSEEk: a fully automated content-based image query system, Proceedings of the fourth ACM international conference on Multimedia, p.87-98, November 18-22, 1996, Boston, Massachusetts, United States [doi>10.1145/244130.244151]
• 10: Kang, H., Shneiderman, B.: Visualization Methods for Personal Photo Collections: Browsing and Searching in the PhotoFinder. In Proc. IEEE ICME 2000. 1539--1542
• 11: Direct Annotation: A Drag-and-Drop Strategy for Labeling Photos, Proceedings of the International Conference on Information Visualisation, p.88, July 19-21, 2000
• 12: Madden, M.: America's Online Pursuits: The changing picture of who's online and what they do. Pew Internet & American life project. www.pewinternet.org. 2003.
• 13: Minolta Ltd. DiMAGE Messenger. http://www.dimagemessenger.com/
• 14: Picasa Inc. Hello. http://www.picasa.net/
• 15: Levenshtein, V. I.: Binary codes capable of correcting deletions, insertions and reversals. Doklady Akademii Nauk SSSR 163(4), 1965. 845--848
• 16: Montague, R.: Formal Philosophy; Selected papers of Richard Montague. Yale University Press, New Haven. (1974).
• 17: Radford, A.: Syntactic theory and the structure of English. Cambridge University Press (1997).
• 18: Rosetta, M. T.: Compositional Translation. Kluwer International Series in Engineering and computer science: Volume 273. (1994).
• 19: Moortgat, M.: Categorial investigations. University of Amsterdam (1988).
• 20: Bob Carpenter, Type-logical semantics, MIT Press, Cambridge, MA, 1998
• 21: Bunt, H: Dynamic Interpretation and Dialogue Theory. In M. M. Taylor, D. G. Bouwhuis, F. Neel(eds.): The Structure of Multimodal Dialogue II. John Benjamins. 1998.
• 22: Ronnie W. Smith , D. Richard Hipp, Spoken natural language dialog systems: a practical approach, Oxford University Press, Inc., New York, NY, 1995
• 23: Frederking, R. E.: Integrated Natual Language Dialogue: A Computational Model. Kluwer Academic Publishers. 1988.
• 24: Freeman, J., Lessiter, IJsselsteijn, W. A.: An Introduction to Presence: A Sense of Being There in a Mediated Environment. The Psychologist: April 2001.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications
          Subjects: Image databases

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Natural language; Prototyping
      H.5.3 Group and Organization Interfaces
          Subjects: Collaborative computing

General Terms:
Design, Human Factors, Languages, Management

Keywords:
annotation, digital photos, instant messaging systems, metadata