Real-time new event detection for video streams

ABSTRACT

Online detection of video clips that present previously unseen events in a video stream is still an open challenge to date. For this online new event detection (ONED) task, existing studies mainly focus on optimizing the detection accuracy instead of the detection efficiency. As a result, it is difficult for existing systems to detect new events in real time, especially for large-scale video collections such as the video content available on the Web. In this paper, we propose several scalable techniques to improve the video processing speed of a baseline ONED system by orders of magnitude without sacrificing much detection accuracy. First, we use text features alone to filter out most of the non-new-event clips and to skip those expensive but unnecessary steps including image feature extraction and image similarity computation. Second, we use a combination of indexing and compression methods to speed up text processing. We implemented a prototype of our optimized ONED system on top of IBM's System S. The effectiveness of our techniques is evaluated on the standard TRECVID 2005 benchmark, which demonstrates that our techniques can achieve a 480-fold speedup with detection accuracy degraded less than 5%.

REFERENCES

• 1: L. Agnihotri, N. Dimitrova, and T. McGee et al. Envolvable Visual Commercial Detector. CVPR (2) 2003: 79--84.
• 2: James Allan , Victor Lavrenko , Hubert Jin, First story detection in TDT is hard, Proceedings of the ninth international conference on Information and knowledge management, p.374-381, November 06-11, 2000, McLean, Virginia, United States [doi>10.1145/354756.354843]
• 3: James Allan , Ron Papka , Victor Lavrenko, On-line new event detection and tracking, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.37-45, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.290954]
• 4: Thorsten Brants , Francine Chen , Ayman Farahat, A System for new event detection, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860495]
• 5: R. Braun, R. Kaneshiro. Exploiting Topic Pragmatics for New Event Detection in TDT-2004. TDT-2004 Workshop.
• 6: M. Campbell, S. Ebadollahi, and D. Joshi et al. IBM Research TRECVID-2006 Video Retrieval System. NIST TRECVID workshop, 2006.
• 7: Francine Chen , Ayman Farahat , Thorsten Brants, Story link detection and new event detection are asymmetric, Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology: companion volume of the Proceedings of HLT-NAACL 2003--short papers, p.13-15, May 27-June 01, 2003, Edmonton, Canada [doi>10.3115/1073483.1073488]
• 8: M. Clayton. US Plans Massive Data Sweep. The Christian Science Monitor, February 09, 2006. http://www.csmonitor.com/2006/0209/p01s02-uspo.html, 2006.
• 9: P. Duygulu, M. Chen, and A.G. Hauptmann. Comparison and Combination of Two Novel Commercial Detection Methods. ICME 2004: 1267--1270.
• 10: Pinar Duygulu , Jia-Yu Pan , David A. Forsyth, Towards auto-documentary: tracking the evolution of news stories, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027719]
• 11: W. Hsu, S. Chang. Topic Tracking across Broadcast News Videos with Visual Duplicates and Semantic Concepts. ICIP 2006: 141--144.
• 12: IBM Technology Translates Arabic Media Broadcasts to English. http://www.sda-asia.com/sda/news/psecom,id,11163,srn,4,channel,developer,nodeid,4,_language,Singapore.html#, 2006.
• 13: Kun-Lung Wu , Kirsten W. Hildrum , Wei Fan , Philip S. Yu , Charu C. Aggarwal , David A. George , Buǧra Gedik , Eric Bouillet , Xiaohui Gu , Gang Luo , Haixun Wang, Challenges and experience in prototyping a multi-modal stream analytic and monitoring application on System S, Proceedings of the 33rd international conference on Very large data bases, September 23-27, 2007, Vienna, Austria
• 14: Giridhar Kumaran , James Allan, Text classification and named entities for new event detection, Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, July 25-29, 2004, Sheffield, United Kingdom [doi>10.1145/1008992.1009044]
• 15: John R. Kender , Milind R. Naphade, Visual Concepts for News Story Tracking: Analyzing and Exploiting the NIST TRECVID Video Annotation Experiment, Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 1, p.1174-1181, June 20-26, 2005 [doi>10.1109/CVPR.2005.371]
• 16: Xiaoyan Li , W. Bruce Croft, Novelty detection based on sentence level patterns, Proceedings of the 14th ACM international conference on Information and knowledge management, October 31-November 05, 2005, Bremen, Germany [doi>10.1145/1099554.1099734]
• 17: E. Lipton. Software to Monitor Overseas Opinions of U.S. The New York Times, October 4, 2006. http://news.zdnet.com/2100-9588_22-6122641.html, 2006.
• 18: Gang Luo , Chunqiang Tang , Philip S. Yu, Resource-adaptive real-time new event detection, Proceedings of the 2007 ACM SIGMOD international conference on Management of data, June 11-14, 2007, Beijing, China [doi>10.1145/1247480.1247536]
• 19: Zhiwei Li , Bin Wang , Mingjing Li , Wei-Ying Ma, A probabilistic model for retrospective news event detection, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, August 15-19, 2005, Salvador, Brazil [doi>10.1145/1076034.1076055]
• 20: R. Peterson. IBM Strives for Super Human Speech. http://www.accessible-devices.com/superspeech.html, 2006.
• 21: M. F. Porter. An Algorithm for Suffix Stripping. Program 14(3): 130--137, 1980.
• 22: Nicola Stokes , Joe Carthy, Combining semantic and syntactic document classifiers to improve first story detection, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.424-425, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.384068]
• 23: A. Singhal. Modern Information Retrieval: A Brief Overview. IEEE Data Eng. Bull. 24(4): 35--43, 2001.
• 24: SMART Stopword List. http://www.lextek.com/manuals/onix/stopwords2.html, 2005.
• 25: TDT Homepage. http://www.nist.gov/speech/tests/tdt.
• 26: TREC Video Retrieval Evaluation. http://www-nlpir.nist.gov/projects/trecvid.
• 27: E. M. Voorhees. Overview of TREC 2005. TREC 2005: 1--15.
• 28: X. Wu, C. Ngo, and Q. Li. Threading and Autodocumenting News Videos: a Promising Solution to Rapidly Browse News Topics. IEEE Signal Processing Magazine 23(2): 59--68, 2006.
• 29: Yiming Yang , Tom Pierce , Jaime Carbonell, A study of retrospective and on-line event detection, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.28-36, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.290953]
• 30: Yiming Yang , Jian Zhang , Jaime Carbonell , Chun Jin, Topic-conditioned novelty detection, Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, July 23-26, 2002, Edmonton, Alberta, Canada [doi>10.1145/775047.775150]
• 31: Yun Zhai , Mubarak Shah, Tracking news stories across different sources, Proceedings of the 13th annual ACM international conference on Multimedia, November 06-11, 2005, Hilton, Singapore [doi>10.1145/1101149.1101152]
• 32: YouTube Homepage. http://www.youtube.com.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Information filtering

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.4 Systems and Software
          Subjects: Performance evaluation (efficiency and effectiveness)
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Video (e.g., tape, disk, DVI)

General Terms:
Algorithms, Experimentation, Performance

Keywords:
efficiency, large-scale video streaming, online new event detection, real-time filtering