An efficient manual image annotation approach based on tagging and browsing

ABSTRACT

This paper investigates new approaches to improve the efficiency of manual image annotation and help users to produce better annotation results in a given amount of time. Although important in practice, this issue has rarely been studied in a quantitative way before. To achieve this, we first propose two time models to analyze the annotation process for two popular manual annotation approaches, i.e., tagging and browsing. The complementary properties of these approaches have inspired us to merge them to develop a hybrid annotation algorithms called frequency-based annotation. Our experiments on large-scale multimedia collections have shown that the proposed algorithm can achieve an up to 40% annotation time reduction compared with the baseline methods. In other words, it can produce considerably better results using the same annotation time.

REFERENCES

• 1: Kobus Barnard , Pinar Duygulu , David Forsyth , Nando de Freitas , David M. Blei , Michael I. Jordan, Matching words and pictures, The Journal of Machine Learning Research, 3, 3/1/2003
• 2: G. W. Furnas , T. K. Landauer , L. M. Gomez , S. T. Dumais, The vocabulary problem in human-system communication, Communications of the ACM, v.30 n.11, p.964-971, Nov. 1987 [doi>10.1145/32206.32212]
• 3: C. Halaschek-Wiener, J. Golbeck, A. Schain, M. Grove, B. Parsia, and J. Hendler. Photostuff-an image annotation tool for the semantic web. In Proc. of 4th international semantic web conference, 2005.
• 4: Alexander G. Hauptmann , Wei-Hao Lin , Rong Yan , Jun Yang , Ming-Yu Chen, Extreme video retrieval: joint maximization of human and computer performance, Proceedings of the 14th annual ACM international conference on Multimedia, October 23-27, 2006, Santa Barbara, CA, USA [doi>10.1145/1180639.1180721]
• 5: J. Jeon , V. Lavrenko , R. Manmatha, Automatic image annotation and retrieval using cross-media relevance models, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860459]
• 6: Lyndon S. Kennedy , Shih-Fu Chang , Igor V. Kozintsev, To search or to label?: predicting the performance of search-based automatic image classifiers, Proceedings of the 8th ACM international workshop on Multimedia information retrieval, October 26-27, 2006, Santa Barbara, California, USA [doi>10.1145/1178677.1178712]
• 7: J. Kustanowitz and B. Shneiderman. Motivating annotation for personal digital photo libraries: Lowering barriers while raising incentives. Technical report, HCIL, Univ. of Maryland, 2004.
• 8: Jia Li , James Z. Wang, Real-time computerized annotation of pictures, Proceedings of the 14th annual ACM international conference on Multimedia, October 23-27, 2006, Santa Barbara, CA, USA [doi>10.1145/1180639.1180841]
• 9: Henry Lieberman , Elizabeth Rozenweig , Push Singh, Aria: An Agent for Annotating and Retrieving Images, Computer, v.34 n.7, p.57-62, July 2001 [doi>10.1109/2.933504]
• 10: W.-H. Lin and A. G. Hauptmann. Which thousand words are worth a picture? experiments on video retrieval using a thousand concepts. In Proceedings of IEEE International Conference On Multimedia and Expo (ICME), 2006.
• 11: Milind Naphade , John R. Smith , Jelena Tesic , Shih-Fu Chang , Winston Hsu , Lyndon Kennedy , Alexander Hauptmann , Jon Curtis, Large-Scale Concept Ontology for Multimedia, IEEE MultiMedia, v.13 n.3, p.86-91, July 2006 [doi>10.1109/MMUL.2006.63]
• 12: P. Over, T. Ianeva, W. Kraaij, and A. F. Smeaton. Trecvid 2006 overview. In {NIST} TRECVID-2006, 2006.
• 13: Timo Volkmer , John R. Smith , Apostol (Paul) Natsev, A web-based system for collaborative annotation of large image and video collections: an evaluation and user study, Proceedings of the 13th annual ACM international conference on Multimedia, November 06-11, 2005, Hilton, Singapore [doi>10.1145/1101149.1101341]
• 14: Luis von Ahn , Laura Dabbish, Labeling images with a computer game, Proceedings of the SIGCHI conference on Human factors in computing systems, p.319-326, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985733]
• 15: Xin-Jing Wang , Lei Zhang , Feng Jing , Wei-Ying Ma, AnnoSearch: Image Auto-Annotation by Search, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.1483-1490, June 17-22, 2006 [doi>10.1109/CVPR.2006.58]
• 16: L. Wenyin, S. Dumais, Y. Sun, H. Zhang, M. Czerwinski, and B. Field. Semi-automatic image annotation. In Interact: Conference on HCI, 2001.
• 17: Anita Wilhelm , Yuri Takhteyev , Risto Sarvas , Nancy Van House , Marc Davis, Photo annotation on a camera phone, CHI '04 extended abstracts on Human factors in computing systems, April 24-29, 2004, Vienna, Austria [doi>10.1145/985921.986075]
• 18: Yiming Yang , Jan O. Pedersen, A Comparative Study on Feature Selection in Text Categorization, Proceedings of the Fourteenth International Conference on Machine Learning, p.412-420, July 08-12, 1997

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Indexing methods

General Terms:
Algorithms, Performance, Theory

Keywords:
browsing, image annotation, tagging