Spatio-temporal features for robust content-based video copy detection

ABSTRACT

n this paper, we present a new method for robust content-based video copy detection based on local spatio-temporal features. As we show by experimental validation, the use of local spatio-temporal features instead of purely spatial ones brings additional robustness and discriminativity. Efficient operation is ensured by using the new spatio-temporal features proposed in [20]. To cope with the high-dimensionality of the resulting descriptors, these features are incorporated in a disk-based index and query system based on p-stable locality sensitive hashing. The system is applied to the task of video footage reuse detection in news broadcasts. Results are reported on 88 hours of news broadcast data from the TRECVID2006 dataset.

REFERENCES

• 1: H. Bay, T. Tuytelaars, and L. Van Gool. Surf: Speeded-up robust features. In Proceedings of the 9th European Conference on Computer Vision, 2006.
• 2: Kristleifur Dadason , Herwig Lejsek , Fridrik Ásmundsson , Björn Jónsson , Laurent Amsaleg, Videntifier: identifying pirated videos in real-time, Proceedings of the 15th international conference on Multimedia, September 25-29, 2007, Augsburg, Germany [doi>10.1145/1291233.1291346]
• 3: Mayur Datar , Nicole Immorlica , Piotr Indyk , Vahab S. Mirrokni, Locality-sensitive hashing scheme based on p-stable distributions, Proceedings of the twentieth annual symposium on Computational geometry, June 08-11, 2004, Brooklyn, New York, USA [doi>10.1145/997817.997857]
• 4: C. Harris and M. Stephens. A Combined Corner and Edge Detector. In 4th ALVEY Vision Conference, pages 147--151, 1988.
• 5: X.-S. Hua, X. Chen, and H.-J. Zhang. Robust video signature based on ordinal measure. ICIP'04, 1:685--688 Vol. 1, Oct. 2004.
• 6: Piotr Indyk , Rajeev Motwani, Approximate nearest neighbors: towards removing the curse of dimensionality, Proceedings of the thirtieth annual ACM symposium on Theory of computing, p.604-613, May 24-26, 1998, Dallas, Texas, United States [doi>10.1145/276698.276876]
• 7: A. Joly, O. Buisson, and C. Frelicot. Content-based copy retrieval using distortion-based probabilistic similarity search. Multimedia, IEEE Transactions on, 9(2):293--306, Feb. 2007.
• 8: B. Jonsson, H. Lejsek, and L. Amsaleg. The eff2 project: Towards efficient and effective support for large-scale high-dimensional indexing. Content-Based Multimedia Indexing, 2007. CBMI '07. International Workshop on, pages 1--10, June 2007.
• 9: Yan Ke , Rahul Sukthankar , Larry Huston, An efficient parts-based near-duplicate and sub-image retrieval system, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027729]
• 10: C. Kim and B. Vasudev. Spatiotemporal sequence matching for efficient video copy detection. Circuits and Systems for Video Technology, IEEE Transactions on, 15(1):127--132, Jan. 2005.
• 11: Ivan Laptev, On Space-Time Interest Points, International Journal of Computer Vision, v.64 n.2-3, p.107-123, September 2005 [doi>10.1007/s11263-005-1838-7]
• 12: J. Law-To, O. Buisson, V. Gouet-Brunet, and N. Boujemaa. Robust voting algorithm based on labels of behavior for video copy detection. October 2006.
• 13: Y. Li, J. Jin, and X. Zhou. Video matching using binary signature. ISPACS'05, pages 317--320, Dec. 2005.
• 14: David G. Lowe, Distinctive Image Features from Scale-Invariant Keypoints, International Journal of Computer Vision, v.60 n.2, p.91-110, November 2004 [doi>10.1023/B:VISI.0000029664.99615.94]
• 15: K. Mikolajczyk , T. Tuytelaars , C. Schmid , A. Zisserman , J. Matas , F. Schaffalitzky , T. Kadir , L. Van Gool, A Comparison of Affine Region Detectors, International Journal of Computer Vision, v.65 n.1-2, p.43-72, November 2005 [doi>10.1007/s11263-005-3848-x]
• 16: Mihai Osian , Luc Van Gool, Video shot characterization, Machine Vision and Applications, v.15 n.3, p.172-177, July 2004 [doi>10.1007/s00138-004-0141-x]
• 17: Sébastien Poullot , Olivier Buisson , Michel Crucianu, Z-grid-based probabilistic retrieval for scaling up content-based copy detection, Proceedings of the 6th ACM international conference on Image and video retrieval, p.348-355, July 09-11, 2007, Amsterdam, The Netherlands [doi>10.1145/1282280.1282334]
• 18: Josef Sivic , Andrew Zisserman, Video Google: A Text Retrieval Approach to Object Matching in Videos, Proceedings of the Ninth IEEE International Conference on Computer Vision, p.1470, October 13-16, 2003
• 19: C. Tomasi and T. Kanade. Detection and tracking of point features. Technical Report CMU-CS--91--132, Carnegie Mellon University, April 1991.
• 20: G. Willems, T. Tuytelaars, and L. Van Gool. An efficient dense and scale-invariant spatio-temporal interest point detector. In Proceedings of the 10th European Conference on Computer Vision, 2008.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

General Terms:
Algorithms, Experimentation

Keywords:
content-based video copy detection