A generalised cross-modal clustering method applied to multimedia news semantic indexing and retrieval

ABSTRACT

Current Web technology has enabled the distribution of informative content through dynamic media platforms.

In addition, the availability of the same content in the form of digital multimedia data has dramatically increased.

Content-based, cross-media retrieval applications are needed to efficiently access desired information from this variety of data sources. This paper presents a novel approach for cross-media information aggregation, and describes a prototype system implementing this approach. The prototype adopts online newspaper articles and TV newscasts as information sources, to deliver a service made up of items including both contributions. Extensive experiments prove the effectiveness of the proposed approach in a real-world business context.

REFERENCES

• 1: Jae-wook Ahn , Peter Brusilovsky , Jonathan Grady , Daqing He , Sue Yeon Syn, Open user profiles for adaptive news systems: help or harm?, Proceedings of the 16th international conference on World Wide Web, May 08-12, 2007, Banff, Alberta, Canada [doi>10.1145/1242572.1242575]
• 2: J. Arlandis, P. Over, and W. Kraaij. Boundary Error Analysis and Categorization in the TRECVID News Story Segmentation Task. In Proc. of CIVR05, pages 103--112, 2005.
• 3: R. Basili, M. Cammisa, and E. Donati. RitroveRAI: A Web Application for Semantic Indexing and Hyperlinking of Multimedia News. In Proc. of the Intl. Semantic Web Conf., pages 97--111, 2005.
• 4: Levent Bolelli , Seyda Ertekin , Ding Zhou , C. Lee Giles, K-SVMeans: A Hybrid Clustering Algorithm for Multi-Type Interrelated Datasets, Proceedings of the IEEE/WIC/ACM International Conference on Web Intelligence, p.198-204, November 02-05, 2007 [doi>10.1109/WI.2007.68]
• 5: F. Brugnara, M. Cettolo, M. Federico, and D. Giuliani. A system for the segmentation and transcription of Italian radio news. In Proc. of RIAO, Content-Based Multimedia Information Access, 2000.
• 6: Abhinandan S. Das , Mayur Datar , Ashutosh Garg , Shyam Rajaram, Google news personalization: scalable online collaborative filtering, Proceedings of the 16th international conference on World Wide Web, May 08-12, 2007, Banff, Alberta, Canada [doi>10.1145/1242572.1242610]
• 7: K. Deschacht and M.-F. Moens. Finding the Best Picture: Cross-Media Retrieval of Content. In Proc. of ECIR 2008, pages 539--546, 2008.
• 8: Marco Di Iulio , Alberto Messina, Use of Probabilistic Clusters Supports for Broadcast News Segmentation, Proceedings of the 2008 19th International Conference on Database and Expert Systems Application, p.600-604, September 01-05, 2008 [doi>10.1109/DEXA.2008.65]
• 9: M. Farr¶us, P. Ejarque, A. Temko, and J. Hernando. Histogram equalization in svm multimodal person verification. In Proc. of the Intl. Conf. on Advances in Biometrics, pages 819--827, 2007.
• 10: Monika Henzinger , Bay-Wei Chang , Brian Milch , Sergey Brin, Query-free news search, Proceedings of the 12th international conference on World Wide Web, May 20-24, 2003, Budapest, Hungary [doi>10.1145/775152.775154]
• 11: Vipul Kashyap , Amit Sheth, Semantic and schematic similarities between database objects: a context-based approach, The VLDB Journal — The International Journal on Very Large Data Bases, v.5 n.4, p.276-304, December 1996 [doi>10.1007/s007780050029]
• 12: Xin Li , Jun Yan , Zhihong Deng , Lei Ji , Weiguo Fan , Benyu Zhang , Zheng Chen, A novel clustering-based RSS aggregator, Proceedings of the 16th international conference on World Wide Web, May 08-12, 2007, Banff, Alberta, Canada [doi>10.1145/1242572.1242824]
• 13: Alberto Messina, An Application of NLP and Audiovisual Content Analysis for Integration of Multimodal Databases of Current Events, Proceedings of the 13th international conference on Natural Language and Information Systems: Applications of Natural Language to Information Systems, June 24-27, 2008, London, UK [doi>10.1007/978-3-540-69858-6_41]
• 14: A. Messina , R. Borgotallo , G. Dimino , D. Airola Gnota , L. Boch, ANTS: A Complete System for Automatic News Programme Annotation Based on Multimodal Analysis, Proceedings of the 2008 Ninth International Workshop on Image Analysis for Multimedia Interactive Services, p.219-222, May 07-09, 2008 [doi>10.1109/WIAMIS.2008.15]
• 15: H. T. Pao, Y. Y. Xu, S. C. Chung, and H. C. Fu. Constructing and application of multimedia tv news archives. In Intl. Workshop on Multimedia Content Analysis and Mining, pages 151--160, 2007.
• 16: Canhui Wang , Min Zhang , Shaoping Ma , Liyun Ru, Automatic online news issue construction in web environment, Proceeding of the 17th international conference on World Wide Web, April 21-25, 2008, Beijing, China [doi>10.1145/1367497.1367560]
• 17: David Webster , Weihong Huang , Darren Mundy , Paul Warren, Context-orientated news riltering for web 2.0 and beyond, Proceedings of the 15th international conference on World Wide Web, May 23-26, 2006, Edinburgh, Scotland [doi>10.1145/1135777.1135985]
• 18: Xiao Wu , Jintao Li , Yongdong Zhang , Sheng Tang , Shi-Yong Neo, Personalized multimedia web summarizer for tourist, Proceeding of the 17th international conference on World Wide Web, April 21-25, 2008, Beijing, China [doi>10.1145/1367497.1367638]
• 19: C. Xu, J. Wang, H. Lu, and Y. Zhang. A novel framework for semantic annotation and personalized retrieval of sports video. IEEE Trans. on Multimedia, 10(3):421--436, 2008.
• 20: Y. Zhang, C. Xu, Y. Rui, J. Wang, and H. Lu. Semantic event extraction from basketball games using multi-modal analysis. In Proc. of the ICME 2007, pages 2190--2193, 2007.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.0 GENERAL

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Linguistic processing

General Terms:
Algorithms, Experimentation

Keywords:
cross-modal clustering, multimedia mashup, news retrieval