Take a look at the new version of this page: [ beta version ]. Tell us what you think.

ABSTRACT

Today's rapid developments in digital media processing capabilities, and network speeds, make the dissemination of multimedia data extremely rapid and reliable, and have attracted significant research attention to video analysis, event detection, tracking and surveillance. In this work, a novel, generally applicable approach to the detection of human activity in video is presented. The areas of activity in the video are first detected via the accumulation and statistical processing of the motion vectors in all frames. The times (frames) at which events begin and end are defined as moments at which the statistical distribution of the motion vectors changes, for each pixel. These time instants are estimated in a novel manner, by applying sequential likelihood ratio testing on the motion vectors of the pixels that have been found to be active.

The proposed system provides a theoretically sound solution for the detection of temporal changes in the human (or other) activity in video, without resorting to use of prior knowledge, heuristics, or ad-hoc thresholds. Sequential detection techniques allow us to find the frames where events begin and end, but also allows to pre-define the desired probabilities of false alarm and miss for the system. This is entirely novel for the temporal localization of activities and events in the video processing literature. Finally, sequential change detection methods require the smallest number of samples to detect a change, so they ensure the fastest detection of events. Experiments are performed with real sequences, involving human activities, for varying probabilities of false alarm and miss. Comparison with ground truth results shows that, indeed, the proposed method leads to meaningful localization of events both in time and in space.

REFERENCES

• 1: H. A. Shot-boundary detection:unraveled and resolved? IEEE Transactions on Circuits and Systems for Video Technology 12(2):90--105, 2002.
• 2: Ernesto L. Andrade , Scott Blunsden , Robert B. Fisher, Hidden Markov Models for Optical Flow Analysis in Crowds, Proceedings of the 18th International Conference on Pattern Recognition, p.460-463, August 20-24, 2006 [doi>10.1109/ICPR.2006.621]
• 3: J. Barron and R. Eagleson. Recursive estimation of time-varying motion and structure parameters. Pattern Recognition 29(5):797--818, Dec. 1996.
• 4: M. J. Black and P. Anandan. A framework for the robust estimation of optical flow. In Proc. IEEE 4th Int. Conf. Computer Vision pages 231--236, May 1993.
• 5: Aaron F. Bobick , James W. Davis, The Recognition of Human Movement Using Temporal Templates, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.23 n.3, p.257-267, March 2001 [doi>10.1109/34.910878]
• 6: J. Bouguet. Image sequence enhancement using multiple motions analysis. In Intel Corporation. Microprocessor Research Labs
• 7: Alexia Briassouli , Stamatia Dasiopoulou , Ioannis Kompatsiaris, Ontology-Based Trajectory Analysis for Semantic Event Detection, Proceedings of the International Conference on Semantic Computing, p.735-742, September 17-19, 2007 [doi>10.1109/ICSC.2007.66]
• 8: Alexia Briassouli , Vasileios Mezaris , Ioannis Kompatsiaris, Color aided motion-segmentation and object tracking for video sequences semantic analysis: Articles, International Journal of Imaging Systems and Technology, v.17 n.3, p.174-189, October 2007 [doi>10.1002/ima.v17:3]
• 9: A. Briassouli, V. Mezaris, and I. Kompatsiaris. Video segmentation and semantics extraction from the fusion of motion and color information.I n Proceedings of the 2007 IEEE International Conference on Image Processing, ICIP 2007 pages 2014--2017, San Antonio, Texas, Sept. 2007.
• 10: P. A. Delaney. Signal detection using third-order moments. Circuits Systems Signal Process 13(4):481--496, 1994.
• 11: F. Dufaux and J. Konrad. Efficient, robust,and fast global motion estimation for video coding. IEEE Transactions on Image Processing 9(3): 497Ű501, March 2000.
• 12: W. Feller. An Introduction to Probability Theory and its Applications volume 1. John Wiley & Sons, New York, 1966.
• 13: M. Ghallab. On chronicles: Representation, on-line recognition and learning. In D. Aiello and Shapiro, editors, Proc. Principles of Knowledge Representation and Reasoning pages 597--606, Nov. 1996.
• 14: Rafael C. Gonzalez , Richard E. Woods, Digital Image Processing, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 2001
• 15: Lowell Jacobson , Harry Wechsler, Derivation of optical flow using a spatiotemporal-frequency approach, Computer Vision, Graphics, and Image Processing, v.38 n.1, p.29-65, April 1987 [doi>10.1016/S0734-189X(87)80152-4]
• 16: Sooyeong Kwak , Guntae Bae , Kilcheon Kim , Hyeran Byun, Unusual Event Recognition for Mobile Alarm System, Proceedings of the 7th international conference on Computational Science, Part IV: ICCS 2007, p.417-424, May 27-30, 2007, Beijing, China [doi>10.1007/978-3-540-72590-9_57]
• 17: J. L. Barron, D. J. Fleet, S. S. Beauchemin, and T. A. Burkitt. Performance of optical flow techniques. In 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition pages 236--242, June 1992.
• 18: A. Marana, M. Cavenaghi, R. Ulson, and F. Drumond. Real-time crowd density estimation using images. In ISVC05 pages 355--362, 2005.
• 19: A. Nandi. Robust estimation of third-order cumulants in applications ofhigher-order statistics. Radar and Signal Processing, IEE Proceedings 140(6):380--389, Dec.1993.
• 20: I. Nikiforov. A generalized change detection problem. IEEE Transactions on Information Theory 41(1):171--187, Jan. 1995.
• 21: A. Papoulis. Probability, Random Variables, and Stochastic Processes McGraw-Hill, New York, 2nd edition, 1987.
• 22: J. K. Patel and C. B. Read. Handbook of the Normal Distribution Dekker, New York, 1982.
• 23: P. Z. Peebles. Probability, Random Variables and Random Signal Principles McGraw-Hill Inc, Boston, 2001.
• 24: H. Vincent Poor, An introduction to signal detection and estimation (2nd ed.), Springer-Verlag New York, Inc., New York, NY, 1994
• 25: H. Rahmalan, M. Nixon, and J. Carter. On crowd density estimation for surveillance. In Crime and Security, 2006. The Institution of Engineering and Technology Conference on pages 540--545, June 2006.
• 26: C. S. Regazzoni, C. Sacchi, A. Teschioni, and S. Giulini. Higher-order-statistics-based sharpness evaluation of a generalized gaussian pdf model in impulsive noisy environments. In Statistical Signal and Array Processing, 1998. Proceedings., Ninth IEEE SP Workshop on pages 411--414, Sept. 1998.
• 27: E. Stringa and C. S. Regazzoni. Content-based retrieval and real time detection from video sequences acquired by surveillance systems. In Image Processing, 1998. ICIP 98. Proceedings. 1998 International Conference on volume 3, pages 138--142, Oct. 1998.
• 28: A. Wald. Sequential Analysis Dover Publications, 2004.
• 29: J. Weijer and T. Gevers. Robust optical flow from photometric invariants. In Proc. IEEE International Conference on Image Processing, 2004 volume3, pages 1835--1838, Oct. 2004.
• 30: M. Welling. Robust higher order statistics. In Tenth International Workshop on Artificial Intelligence and Statistics pages 405--412, Barbados, Jan. 2005.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.4 INFORMATION SYSTEMS APPLICATIONS
      H.4.m Miscellaneous

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
        D.2.10 Design**

General Terms:
Algorithms

Keywords:
activity localization, sequential detection, video surveillance