Scene duplicate detection based on the pattern of discontinuities in feature point trajectories

ABSTRACT

The paper is aiming to detect and retrieve videos of the same scene (scene duplicates) from broadcast video archives. Scene duplicate is composed of different pieces of footage of the same scene, the same event, at the same time, but from the different viewpoints. Scene duplicate detection would be particularly useful to identify the same event reported in different programs from different broadcast stations. The approach should be invariant to viewpoint changes. We focused on object motion in videos and devised a video matching approach based on the temporal pattern of discontinuities obtained from feature point trajectories. We developed an acceleration method based on the discontinuity pattern, which is more robust to variations in camerawork and editing than conventional features, to dramatically reduce the computation burden. We compared our approach with an existing video matching method based on the local feature of keyframe. The spatial registration strategy of this method was also used with the proposed approach to cope with visually different unrelated video pairs. The performance and effectiveness of our approach was demonstrated on actual broadcasted videos.

REFERENCES

• 1: Yaron Caspi , Michal Irani, Spatio-Temporal Alignment of Sequences, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.11, p.1409-1424, November 2002 [doi>10.1109/TPAMI.2002.1046148]
• 2: Daniel DeMenthon , David Doermann, Video retrieval using spatio-temporal descriptors, Proceedings of the eleventh ACM international conference on Multimedia, November 02-08, 2003, Berkeley, CA, USA [doi>10.1145/957013.957124]
• 3: Ling-Yu Duan , Jinqiao Wang , Yantao Zheng , Jesse S. Jin , Hanqing Lu , Changsheng Xu, Segmentation, categorization, and identification of commercial clips from TV streams using multimodal analysis, Proceedings of the 14th annual ACM international conference on Multimedia, October 23-27, 2006, Santa Barbara, CA, USA [doi>10.1145/1180639.1180697]
• 4: A. Hampapur and R. M. Bolle. Comparison of distance measures for video copy detection. In ICME, 2001.
• 5: W. H. Hsu and S.-F. Chang. Topic tracking across broadcast news videos with visual duplicates and semantic concepts. In ICIP, pages 141--144, 2006.
• 6: I. Ide, H. Mo, N. Katayama, and S. Satoh. Topic threading for structuring a large-scale news video archive. In CIVR, pages 123--131, 2004.
• 7: K. Iwamoto, E. Kasutani, and A. Yamada. Image signature robust to caption superimposition for video sequence identification. In ICIP, pages 3185--3188, 2006.
• 8: Alejandro Jaimes , Shih-Fu Chang , Alexander C. Loui, Duplicate detection in consumer photography and news video, Proceedings of the tenth ACM international conference on Multimedia, December 01-06, 2002, Juan-les-Pins, France [doi>10.1145/641007.641098]
• 9: A. Joly, O. Buisson, and C. Frelicot. Content-based copy retrieval using distortion-based probabilistic similarity search. IEEE Transactions on Multimedia, 9(2):293--306, February 2007.
• 10: Julien Law-To , Olivier Buisson , Valerie Gouet-Brunet , Nozha Boujemaa, Robust voting algorithm based on labels of behavior for video copy detection, Proceedings of the 14th annual ACM international conference on Multimedia, October 23-27, 2006, Santa Barbara, CA, USA [doi>10.1145/1180639.1180826]
• 11: Herwig Lejsek , Fridrik H. Ásmundsson , Björn Thór Jónsson , Laurent Amsaleg, Scalability of local image descriptors: a comparative study, Proceedings of the 14th annual ACM international conference on Multimedia, October 23-27, 2006, Santa Barbara, CA, USA [doi>10.1145/1180639.1180760]
• 12: David G. Lowe, Distinctive Image Features from Scale-Invariant Keypoints, International Journal of Computer Vision, v.60 n.2, p.91-110, November 2004 [doi>10.1023/B:VISI.0000029664.99615.94]
• 13: Chong-Wah Ngo , Wan-Lei Zhao , Yu-Gang Jiang, Fast tracking of near-duplicate keyframes in broadcast domain with transitivity propagation, Proceedings of the 14th annual ACM international conference on Multimedia, October 23-27, 2006, Santa Barbara, CA, USA [doi>10.1145/1180639.1180827]
• 14: S. Satoh. News video analysis based on identical shot detection. In IEEE International Conference on Multimedia and Expo, pages 69--72, 2002.
• 15: Shin'ichi Satoh , Masao Takimoto , Jun Adachi, Scene duplicate detection from videos based on trajectories of feature points, Proceedings of the international workshop on Workshop on multimedia information retrieval, September 24-29, 2007, Augsburg, Bavaria, Germany [doi>10.1145/1290082.1290115]
• 16: Eli Shechtman , Michal Irani, Space-Time Behavior Based Correlation, Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 1, p.405-412, June 20-26, 2005 [doi>10.1109/CVPR.2005.328]
• 17: M. Takimoto, S. Satoh, and M. Sakauchi. Identification and detection of the same scene based on flash light patterns. In ICME, pages 9--12, 2006.
• 18: C. Tomasi and T. Kanade. Detection and tracking of point features. Carnegie Mellon University Technical Report CMU-CS-91-132, April 1991.
• 19: X. Wu, A. G. Hauptmann, and C.-W. Ngo. Novelty and redundancy detection with multimodalities in cross-lingual broadcast domain. Computer Vision and Image Understanding, to appear.
• 20: X. Wu, C.-W. Ngo, and Q. Li. Threading and autodocumenting news videos: a promising solution to rapidly browse news topics. IEEE Signal Processing Magazin, 23(2):59--68, March 2006.
• 21: F. Yamagishi, S. Satoh, and M. Sakauchi. A news video browser using identical video segment detection. In PCM, pages 205--212, 2004.
• 22: Junsong Yuan , Ling-Yu Duan , Qi Tian , Changsheng Xu, Fast and robust short video clip search using an index structure, Proceedings of the 6th ACM SIGMM international workshop on Multimedia information retrieval, October 15-16, 2004, New York, NY, USA [doi>10.1145/1026711.1026722]
• 23: Yun Zhai , Mubarak Shah, Tracking news stories across different sources, Proceedings of the 13th annual ACM international conference on Multimedia, November 06-11, 2005, Hilton, Singapore [doi>10.1145/1101149.1101152]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Indexing methods

Additional Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.9 Applications

General Terms:
Algorithms, Experimentation

Keywords:
feature point tracking, scene duplicate detection, time series analysis, video matching