SmartAlbum: a multi-modal photo annotation system

ABSTRACT

This demonstration presents a novel application (called SmartAlbum) for photo indexing and retrieval that unifies two different image indexing approaches. The system uses two modalities to extract information about a digital photograph; i.e. content-based and speech annotation for image description. The result is a powerful image retrieval tool that has capabilities beyond what current single-mode retrieval systems can offer. We show on a corpus of 1200 images the interest of our approach.

REFERENCES

• 1: Myron Flickner , Harpreet Sawhney , Wayne Niblack , Jonathan Ashley , Qian Huang , Byron Dom , Monika Gorkani , Jim Hafner , Denis Lee , Dragutin Petkovic , David Steele , Peter Yanker, Query by Image and Video Content: The QBIC System, Computer, v.28 n.9, p.23-32, September 1995 [doi>10.1109/2.410146]
• 2: A. Pentland, R. Picard, and S. Sclaroff. Photobook: Content-based Manipulation of Image Databases. In SPIE Storage and Retrieval for Image and Video Databases II, number 2185, Feb 1994.
• 3: Chi-Ren Shyu , Avi Kak , Carla E. Brodley , Lynn S. Broderick, Testing for Human Perceptual Categories in a Physician-in-the-loop CBIR System for Medical Imagery, Proceedings of the IEEE Workshop on Content-Based Access of Image and Video Libraries, p.102, June 22-22, 1999
• 4: T. Tan and P. Mulhem, Image Query System Using Object Probes, in Proc. of the International Conference on Image Processing, ICIP 2001, Oct 2001, Thessaloniki, Greece, pp. 701--704.
• 5: Amanda Stent , Alexander Loui, Using event segmentation to improve indexing of consumer photographs, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.59-65, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.383960]
• 6: Jiayi Chen , Tele Tan , Philippe Mulhem, A Method for Photograph Indexing Using Speech Annotation, Proceedings of the Second IEEE Pacific Rim Conference on Multimedia: Advances in Multimedia Information Processing, p.867-872, October 24-26, 2001
• 7: R.K. Srihari et al, Multimedia Indexing and Retrieval of Voice-Annotated Consumer Photos, in Proc. of the Multimedia Indexing and Retrieval Workshop, SIGIR '99, University of California, Berkeley, U.S, pp. 1--16.
• 8: Allan Kuchinsky , Celine Pering , Michael L. Creech , Dennis Freeze , Bill Serra , Jacek Gwizdka, FotoFile: a consumer multimedia organization and retrieval system, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.496-503, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303143]