Temporal spectral residual: fast motion saliency detection

ABSTRACT

Saliency detection has attracted much attention in recent years. It aims at locating semantic regions in images for further image understanding. In this paper, we address the issue of motion saliency detection for video content analysis. Inspired by the idea of Spectral Residual for image saliency detection, we propose a new method Temporal Spectral Residual on video slices along X-T and Y-T planes, which can automatically separate foreground motion objects from backgrounds, also with the help of threshold selection and voting schemes. Different from conventional background modeling methods with complex mathematical model, the proposed method is only based on Fourier spectrum analysis, so it is simple and fast. The power of our proposed method is demonstrated in the experiments of four typical videos with different dynamic background.

REFERENCES

• 1: http://homepages.inf.ed.ac.uk/rbf/caviardata1/.
• 2: A. Elgammal, R. Duraiswami, D. Harwood, L. S. Davis, R. Duraiswami, and D. Harwood. Background and foreground modeling using nonparametric kernel density for visual surveillance. In Proceedings of the IEEE, 2002.
• 3: D. Gao and N. Vasconcelos. Discriminant saliency for visual recognition from cluttered scenes. In NIPS, 2004.
• 4: X. Hou and L. Zhang. Saliency detection: A spectral residual approach. CVPR, 2007.
• 5: L. Itti and C. Koch. A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Research, 2000.
• 6: L. Itti and C. Koch. Computational modelling of visual attention. Nature Review Neuroscience, 2001.
• 7: L. Itti, C. Koch, and E. Niebur. A model of saliency-based visual attention for rapid scene analysis. PAMI, 1998.
• 8: A. Mittal and N. Paragios. Motion-based background subtraction using adaptive kernel density estimation. CVPR, 2004.
• 9: A. Monnet, A. Mittal, N. Paragios, and V. Ramesh. Background modeling and subtraction of dynamic scenes. ICCV, 2003.
• 10: C. Stauffer and W. Grimson. Adaptive background mixture models for real-time tracking. CVPR, 1999.
• 11: O. Tuzel, F. Porikli, and P. Meer. A bayesian approach to background modeling. In CVPR, 2005.
• 12: D. Walther, L. Itti, M. Riesenhuber, T. Poggio, and C. Koch. Attentional selection for object recognition -- a gentle way. In 2nd Workshop on Biologically Motivated Computer Vision, 2002.
• 13: J. Zhong and S. Sclaroff. Segmenting foreground objects from a dynamic textured background via a robust kalman filter. ICCV, 2003.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.10 Vision and Scene Understanding
            Subjects: Video analysis

General Terms:
Algorithms, Design, Experimentation, Performance

Keywords:
motion saliency detection, temporal spectral residual, video analysis