Unsupervised mesh based segmentation of moving objects

ABSTRACT

Multimedia analysis usually deals with a large amount of video data with a significant number of moving objects. Often it is necessary to reduce the amount of data and to represent the video in terms of moving objects and events. Event analysis can be built on the detection of moving objects.

In order to automatically process a variety of video content in different domain, largely unsupervised moving object segmentation algorithms are needed. We propose a fully unsupervised system for moving object segmentation that does not require any restriction on the video content. Our approach to extract moving objects relies on a mesh-based combination of results from colour segmentation (Mean Shift) and motion segmentation by feature point tracking (KLT tracker).

The proposed algorithm has been evaluated using precision and recall measures for comparing moving objects and their colour segmented regions with manually labelled ground truth data. Results show that the algorithm is comparable to other state-of-the-art algorithms.

The extracted information is used in a search and retrieval tool. For that purpose a moving object representation in MPEG-7 is implemented. It facilitates high performance indexing and retrieval of moving objects and events in large video databases, such as the search for similar moving objects occurring in a certain period.

REFERENCES

• 1: Gianluca Antonini , Santiago Venegas Martinez , Michel Bierlaire , Jean Philippe Thiran, Behavioral Priors for Detection and Tracking of Pedestrians in Video Sequences, International Journal of Computer Vision, v.69 n.2, p.159-180, August 2006 [doi>10.1007/s11263-005-4797-0]
• 2: A. Bab-Hadiashar, N. Gheissari, D. Suter, "Robust Model Based Motion Segmentation", ICPR02, pp. 753--757, Aug. 2002.
• 3: W. Bailer, P. Schallauer, "Detailed audiovisual profile: enabling interoperability between MPEG-7 based systems", International Conference on Multi Media Modelling, 2006.
• 4: W. Bailer, P. Schallauer, H. Bergur Haraldsson, H. Rehatschek, "Optimized Mean Shift Algorithm for Color Segmentation in Image Sequences", Image and Video Communications and Processing, pp. 522--529, 2005.
• 5: W. Bailer, F. Höller, A. Messina, D. Airola, P. Schallauer, M. Hausenblas, "State of the Art of Content Analysis Tools for Video, Audio and Speech", Deliverable Prestospace, Homepage: http://www.prestospace.org/project/deliverables/D15-3_Content_Analysis_Tools.pdf, 2005.
• 6: G. D. Borshukov, G. Bozdagi, Y. Altunbasak, A. M. Tekalp, "Motion segmentation by multistage affine classification", IEEE transactions on Image Processing, Volume: 6, pp: 1591--1594, Nov 1997.
• 7: I. Celasun, A. M. Tekalp, M. H. Gökçetekin, D. M. Harmanc1, "2-D Mesh-Based Video Object Segmentation and Tracking with Occlusion Resolution", Signal Processing: Image Communication Volume 16, Issue 10, August 2001.
• 8: D. Comaniciu, P. Meer, "Robust analysis of feature spaces: colour image segmentation", Department of Electrical and Computer Engineering, 1997.
• 9: John C. Davis, Statistics and Data Analysis in Geology, John Wiley & Sons, Inc., New York, NY, 1990
• 10: M. Donoser, "Object Segmentation in Film and Video", Diploma thesis, TU-Graz, 2003.
• 11: Richard O. Duda , Peter E. Hart , David G. Stork, Pattern Classification (2nd Edition), Wiley-Interscience, 2000
• 12: C. E. Erdem, B. Sankur, "Performance evaluation metrics for object-based video segmentation", Proceedings of the 10th European Signal Processing Conference (EUSIPCO '00), pp. 917--920, Tampere, Finland, September 2000.
• 13: S. Galić, S. Loncarić, "Spatio-temporal image segmentation using optical flow and clustering algorithm", Proceedings of the First International Workshop on Image and Signal Processing and Analysis, 2000.
• 14: E. Galmar, "Survey on Video Object Extraction", Multimedia Talks, Jan. 2005. URL: http://www.eurecom.fr/~galmar/talks/MMtalk_Jan05_slides.pdf
• 15: J. Guo, J. Kim, C.-C. Jay Kuo, "New Video Object Segmentation Technique with Color/Motion Information and Boundary Postprocessing", Applied Intelligence Journal, March 1999.
• 16: Wolfgang Heidrich , Hans-Peter Seidel, Realistic, hardware-accelerated shading and lighting, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.171-178, July 1999 [doi>10.1145/311535.311554]
• 17: B. K. P. Horn, B. G. Schunck, "Determining optical flow", Massachusetts Institute of Technology, 1980.
• 18: R. Lienhart, "Reliable Transition Detection In Videos: A Survey and Practitioner's Guide," International Journal of Image and Graphics (IJIG), vol. 1, No. 3, pp. 469--486, 2001.
• 19: L. Liu, G. Fan, "Combined Key-frame Extraction and Object-based Video Segmentation", IEEE Trans. Circuits and System for Video Technology, 2005.
• 20: B. D. Lucas, T. Kanade, "An Iterative Image Registration Technique with an Application to Stereo Vision", International Joint Conference on Artificial Intelligence, pages 674--679, 1981.
• 21: J. M. Martinez, "MPEG-7 overview", International organisation for standardisation, 2002.
• 22: JungHwan Oh , JeongKyu Lee , Eswar Vemuri, An Efficient Technique for Segmentation of Key Object(s) from Video Shots, Proceedings of the International Conference on Information Technology: Computers and Communications, p.384, April 28-30, 2003
• 23: H. Rehatschek, P. Schallauer, W. Bailer, W. Haas, A. Wertner, "An innovative system for formulating complex combined content-based and keyword-based queries", Proceedings of SPIE-IS&T, Electronic Imaging, vol. 5304, pp. 160--169, 2004.
• 24: G. Tsechpenakis , K. Rapatzikos , N. Tsapatsoulis , S. Kollias, Object tracking in clutter and partial occlusion through rule-driven utilization of Snakes, Proceedings of the 2003 International Conference on Multimedia and Expo - Volume 3 (ICME '03), p.69-72, July 06-09, 2003
• 25: Wei Zeng , Jun Du , Wen Gao , Qingming Huang, Robust moving object segmentation on H.264/AVC compressed video using the block-based MRF model, Real-Time Imaging, v.11 n.4, p.290-299, August 2005 [doi>10.1016/j.rti.2005.04.008]
• 26: Ning Xu , Narendra Ahuja , Ravi Bansal, Object segmentation using graph cuts based active contours, Computer Vision and Image Understanding, v.107 n.3, p.210-224, September, 2007 [doi>10.1016/j.cviu.2006.11.004]
• 27: D. Zhang, G. Lu, "Segmentation of Moving Objects in Image Sequence: A Review", Circuits, Systems and Signal Processing, vol. 20, nr. 2, pp. 143--183, 2001.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis
          Subjects: Color

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Indexing methods

  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.6 Segmentation
      I.4.8 Scene Analysis
          Subjects: Tracking; Shape; Motion

General Terms:
Algorithms, Experimentation, Performance

Keywords:
clustering, displaced frame difference, event detection, mean shift, motion extraction, moving object segmentation, mpeg-7, occlusion, optical flow, spatial segmentation, tracking, unsupervised system