The priority curve algorithm for video summarization

ABSTRACT

In this paper, we introduce the concept of a priority curve associated with a video. We then provide an algorithm that can use the priority curve to create a summary (of a desired length) of any video. The summary thus created exhibits nice continuity properties and also avoids repetition. We have implemented the priority curve algorithm (PCA) and compared it with other summarization algorithms in the literature. We show that PCA is faster than existing algorithms and also produces better quality summaries. The quality of summaries was evaluated by a group of 200 students in Naples, Italy, who watched soccer videos. We also briefly describe a soccer video summarization system we have built on using the PCA architecture and various (classical) image processing algorithms.

REFERENCES

• 1: Sibel Adali , K. Selçuk Candan , Su-Shing Chen , Kutluhan Erol , V. S. Subrahmanian, The advanced video information system: data structures and query processing, Multimedia Systems, v.4 n.4, p.172-186, Aug. 1996 [doi>10.1007/s005300050021]
• 2: N. Ancona, G. Cicirelli, A. Branca, and A. Distante. Goal Detection in Football by Using Support Vector Machines for Classification. Proc. Int. Joint Conference on Neural Networks, Vol. 1, 2001, pp. 611--616.
• 3: S. Ayub and P. Bonissone. Goal Recognition in Complex Domains. IEEE Int, Conf. on Systems, Man, and Cybernetics, Vol. 2, 1994, pp. 1409--1414.
• 4: G. Boccignone, A. Chianese, V. Moscato, and A. Picariello. Foveated Shot Detection for Video Segmentation. to be published in IEEE Trans. on Circuits and Systems for Video Technology, 2004.
• 5: A. Chianese, R. Miscioscia, V. Moscato, S. Parlato, and A. Picariello. A Fuzzy Approach to Video Scene Detection and Its Application For Soccer Matches. to be published in IEEE Intelligent Systems Design and Applications, Budapest, August 2004.
• 6: Daniel DeMenthon , Vikrant Kobla , David Doermann, Video summarization by curve simplification, Proceedings of the sixth ACM international conference on Multimedia, p.211-218, September 13-16, 1998, Bristol, United Kingdom [doi>10.1145/290747.290773]
• 7: M. Fayzullin , V. S. Subrahmanian , A. Picariello , M. L. Sapino, The CPR model for summarizing video, Proceedings of the 1st ACM international workshop on Multimedia databases, November 07-07, 2003, New Orleans, LA, USA [doi>10.1145/951676.951679]
• 8: J. Foote and S. Uchihashi. Summarizing Video Using a Shot Importance Measure and a Frame-Packing Algorithm. Proc. of the Int. Conf. on Acoustics, Speech, and Signal Processing, Phoenix, 1999, Vol. 6, pp. 3041--3044.
• 9: U. Gargi, R. Kasturi, and S. H. Strayer. Performance Characterization of Video-Shot Change Detection Methods. IEEE Trans. on Circuits Systems Video Technology, Vol. 10(1), 2000, pp. 1--13.
• 10: Y. Gong and X. Liu. Video Summarization Using Singular Value Decomposition. Proc. of Computer Vision and Pattern Recognition, 2000, pp. 174--180.
• 11: Rafael C. Gonzales , Paul Wintz, Digital image processing (2nd ed.), Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1987
• 12: A. Hanjalic. Shot-Boundary Detection: Unraveled and Resolved? IEEE Trans. Circuits Systems Video Technology, Vol. 12, 2002) pp. 90--105.
• 13: Liwei He , Elizabeth Sanocki , Anoop Gupta , Jonathan Grudin, Auto-summarization of audio-video presentations, Proceedings of the seventh ACM international conference on Multimedia (Part 1), p.489-498, October 30-November 05, 1999, Orlando, Florida, United States [doi>10.1145/319463.319691]
• 14: R. Lienhart, S. Pfeiffer, and W. Effelsberg. The MoCA Workbench: Support for Creativity in Movie Content Analysis. Proc. IEEE Conf. on Multimedia Computing and Systems, Hiroshima, Japan, 1995, pp. 314--321.
• 15: Howard D. Wactlar , Takeo Kanade , Michael A. Smith , Scott M. Stevens, Intelligent Access to Digital Video: Informedia Project, Computer, v.29 n.5, p.46-52, May 1996 [doi>10.1109/2.493456]
• 16: D. Li and H. Lu. Model Based Video Segmentation. IEEE Trans. Circuits Systems Video Technology, Vol. 5, 1995, pp. 533--544.
• 17: H. Martin and R. Lozano. Dynamic Generation of Video Abstracts Using an Object Oriented Video DBMS. Networking and Information Systems Journal, Vol. 3(1), 2000, pp. 53--75.
• 18: H. R. Naphide and T. S. Huang. A Probabilistic Framework for Semantic Video Indexing, Filtering, and Retrieval. IEEE Transactions on Multimedia, Vol. 3(1), 2001, pp. 141--151.
• 19: E. Oomoto , K. Tanaka, OVID: Design and Implementation of a Video-Object Database System, IEEE Transactions on Knowledge and Data Engineering, v.5 n.4, p.629-643, August 1993 [doi>10.1109/69.234775]
• 20: I. Yahiaoui, B. Merialdo, and B. Huet. Generating Summaries of Multi-Episode Video. IEEE Int. Conf. on Multimedia and Expo, 2001, pp. 22--25.
• 21: Chris Stauffer , W. Eric L. Grimson, Learning Patterns of Activity Using Real-Time Tracking, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.22 n.8, p.747-757, August 2000 [doi>10.1109/34.868677]
• 22: V. S. Subrahmanian, Principles of multimedia database systems, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1998
• 23: Michael J. Swain , Dana H. Ballard, Color indexing, International Journal of Computer Vision, v.7 n.1, p.11-32, Nov. 1991 [doi>10.1007/BF00130487]
• 24: V. Tovinkere and R. J. Qian. Detecting Semantic Events in Soccer Games: Towards a Complete Solution. IEEE Int. Conf. on Multimedia and Expo, 2001, pp. 833--836.
• 25: Ba Tu Truong , Chitra Dorai , Svetha Venkatesh, New enhancements to cut, fade, and dissolve detection processes in video segmentation, Proceedings of the eighth ACM international conference on Multimedia, p.219-227, October 2000, Marina del Rey, California, United States [doi>10.1145/354384.354481]
• 26: D. Zhong and S. -F. Chang. Video Object Model and Segmentation for Content-Based Video Indexing. IEEE Int. Conf. on Circuits and Systems, Hong Kong, 1997.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Information filtering

General Terms:
Algorithms, Design, Performance

Keywords:
probabilistic, summarization, system, video