Semantic-event based analysis and segmentation of wedding ceremony videos

ABSTRACT

Wedding is one of the most important ceremonies in our lives. It symbolizes the birth and creation of a new family. In this paper, we present a system for automatically segmenting a wedding ceremony video into a sequence of recognized wedding events, e.g., the couple's wedding kiss. Our goal is to develop an automatic tool for users to efficiently organize, search, and retrieve his/her treasured wedding memories. Furthermore, the event descriptions could benefit and complement the current research in semantic video understanding. Technically, three kinds of event features, i.e., the speech/music discriminator, flashlight detector, and bride indicator, are exploited to build statistical models for each wedding event. Events are then recognized by a hidden Markov model, which takes into account both the fitness of observed features and the temporal rationality of event ordering to improve the segmentation accuracy. We conducted experiments on a rich set of wedding videos, and the results demonstrate the effectiveness of our approach.

REFERENCES

• 1: L. M. Spangenberg. Timeless Traditions: A Couple's Guide To Wedding Customs Around The World. Universe Publishing, 2001.
• 2: D. Warner. Diane Warner's Contemporary Guide to Wedding Ceremonies. New Page Books, 2006.
• 3: D. Gatica-Perez, A. Loui, and M.-T. Sun. Finding structure in home videos by probabilistic hierarchical clustering. IEEE TCSVT, 13(6):539--548, June 2003.
• 4: Y. Zhai and M. Shah. Automatic segmentation of home videos. Proc. IEEE ICME'05, pages 9--12, 2005.
• 5: Radhakrishna S. V. Achanta , Wei-Qi Yan , Mohan S. Kankanhalli, Modeling Intent for Home Video Repurposing, IEEE MultiMedia, v.13 n.1, p.46-55, January 2006 [doi>10.1109/MMUL.2006.12]
• 6: Z. Rasheed and M. Shah. Detection and representation of scenes in videos. IEEE TMM, 7(6):1097--1105, Dec. 2005.
• 7: P. Yin, X.-S. Hua, and H.-J. Zhang. Automatic time stamp extraction system for home videos. Proc. IEEE ISCAS'02, pages 73--76, 2002.
• 8: Matthew Cooper , Jonathan Foote , Andreas Girgensohn , Lynn Wilcox, Temporal event clustering for digital photo collections, Proceedings of the eleventh ACM international conference on Multimedia, November 02-08, 2003, Berkeley, CA, USA [doi>10.1145/957013.957093]
• 9: Christopher M. Bishop, Pattern Recognition and Machine Learning (Information Science and Statistics), Springer-Verlag New York, Inc., Secaucus, NJ, 2006
• 10: T. Mei, X.-S. Hua, H.-Q. Zhou, and S. Li. Modeling and mining of users' capture intention for home videos. IEEE TMM, 9(1):66--77, Jan. 2007.
• 11: Chitra Dorai , Svetha Venkatesh, Computational Media Aesthetics: Finding Meaning Beautiful, IEEE MultiMedia, v.8 n.4, p.10-12, October 2001 [doi>10.1109/93.959093]
• 12: Aya Aner-Wolf , Lior Wolf, Video de-Abstraction or How to save money on your wedding video, Proceedings of the Sixth IEEE Workshop on Applications of Computer Vision, p.264, December 03-04, 2002
• 13: Yuichiro Takeuchi , Masanori Sugimoto, Video summarization using personal photo libraries, Proceedings of the 8th ACM international workshop on Multimedia information retrieval, October 26-27, 2006, Santa Barbara, California, USA [doi>10.1145/1178677.1178707]
• 14: Yu-Fei Ma , Lie Lu , Hong-Jiang Zhang , Mingjing Li, A user attention model for video summarization, Proceedings of the tenth ACM international conference on Multimedia, December 01-06, 2002, Juan-les-Pins, France [doi>10.1145/641007.641116]
• 15: W.-H. Cheng, C.-W. Wang, and J.-L. Wu. Video adaptation for small display based on content recomposition. IEEE TCSVT, 17(1):43--58, Jan. 2007.
• 16: X.-S. Hua, L. Lu, and H.-J. Zhang. Optimization-based automated home video editing system. IEEE TCSVT, 14(5):572--583, May 2004.
• 17: Gregory D. Abowd , Matthias Gauger , Andreas Lachenmann, The Family Video Archive: an annotation and browsing environment for home movies, Proceedings of the 5th ACM SIGMM international workshop on Multimedia information retrieval, November 07-07, 2003, Berkeley, California [doi>10.1145/973264.973266]
• 18: M. Chen, S.-C. Chen, M.-L. Shyu, and K. Wickramaratna. Semantic event detection via multimodal data mining. IEEE SPM, Mar. 2006.
• 19: J.-H. Lim, Q. Tian, and P. Mulhem. Home photo content modeling for personalized event-based retrieval. IEEE MM, 10(4):28--37, Oct.-Dec. 2003.
• 20: C. Panagiotakis and G. Tziritas. A speech/music discriminator based on rms and zero-crossings. IEEE TMM, 7(1):155--166, Feb. 2005.
• 21: Tong Zhang , C. -C Kuo, Content-Based Audio Classification and Retrieval for Audiovisual Data Parsing, Kluwer Academic Publishers, Norwell, MA, 2001
• 22: Y. Li and C. Dorai. Instructional video content analysis using audio information. IEEE Trans. Audio, Speech, Lang. Process., 14(6):2264--2274, Nov. 2006.
• 23: Rafael C. Gonzalez , Richard E. Woods, Digital Image Processing, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 2001
• 24: B. T. Truong and S. Venkatesh. Determining dramatic intensification via flashing lights in movies. Proc. IEEE ICME'01, pages 61--64, 2001.
• 25: Son Lam Phung , Abdesselam Bouzerdoum , Douglas Chai, Skin Segmentation Using Color Pixel Classification: Analysis and Comparison, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.27 n.1, p.148-154, January 2005 [doi>10.1109/TPAMI.2005.17]
• 26: Thomas M. Cover , Joy A. Thomas, Elements of information theory, Wiley-Interscience, New York, NY, 1991
• 27: Thomas H. Cormen , Clifford Stein , Ronald L. Rivest , Charles E. Leiserson, Introduction to Algorithms, McGraw-Hill Higher Education, 2001

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Indexing methods

Additional Classification:
  H. Information Systems
  H.4 INFORMATION SYSTEMS APPLICATIONS
      H.4.m Miscellaneous

General Terms:
Algorithms

Keywords:
event detection, video segmentation, wedding analysis