3D video surveillance with Augmented Virtual Environments

ABSTRACT

Recent advances in sensing and computing technologies have inspired a new generation of data analysis and visualization systems for video surveillance applications. We present a novel visualization system for video surveillance based on an Augmented Virtual Environment (AVE) that fuses dynamic imagery with 3D models in a real-time display to help observers comprehend multiple streams of temporal data and imagery from arbitrary views of the scene. This paper focuses on our recent technical extensions to our AVE system, including moving object detection, tracking, and 3D display for effective dynamic event comprehension and situational awareness. Moving objects are detected and tracked in video sequences and visualized as pseudo-3D elements in the AVE scene display in real-time. We show results that illustrate the utility and benefits of these new capabilities.

REFERENCES

• 1: Stauffer, C., Grimson, W. E. L. "Adaptive background mixture models for real-time tracking" CVPR 99, Vol. 2, pp. 246--252, June 1999.
• 2: Harville, M., Gordon, G., Woodfill, J. "Foreground Segmentation using Adaptive Mixture Models in Color and Depth". Proceedings of the IEEE Workshop on Detection and Recognition of Events in Video, July 2001.
• 3: Ridder, C., Munklet, O., Kirchner H. "Adaptive background estimation and foreground detection using Kalman-filtering". ICRAM'95, pp. 193--199, 1995.
• 4: Kumar R. Sawhney H. S. Guo Y. Hsu S. Samarasekera. "3D manipulation of motion imagery", ICIP2000, September 2000.
• 5: Remagnino R., Baumberg A., Grove T., Hogg D., Tan T., Worrall A., Baker K. "An integrated traffic and pedestrian Model-based Vision System". Proceedings of BMVC97, volume 2, 8--11th September, pp. 380--389, 1997.
• 6: Dieter Koller , Joseph Weber , Jitendra Malik, Robust multiple car tracking with occlusion reasoning, Proceedings of the third European conference on Computer vision (vol. 1), p.189-196, June 1994, Stocklholm, Sweden
• 7: Ulrich Neumann , Suya You , Jinhui Hu , Bolan Jiang , JongWeon Lee, Augmented Virtual Environments (AVE): Dynamic Fusion of Imagery and 3D Models, Proceedings of the IEEE Virtual Reality 2003, p.61, March 22-26, 2003
• 8: Gonzales, R. C. Woods, R. E. Digital Image Processing. Addison-Wesley, 2nd edition (September 1993), 41--4.
• 9: Spann J. R. Kaufman K. S. "Photogrammetry using 3D graphics and projective textures", IAPRS 2000, Vol. 33.
• 10: Hall B. Trivedi M. "A novel graphical interface and context aware map for incident detection and monitoring", 9th World Congress on Intelligent Transport Systems, October, 2002.
• 11: Tao H. Sawhney H. S. Kumar R. "Dynamic layer representation with applications to tracking", in Proc. IEEE conf. on Computer Vision and Pattern Recognition, CVPR 2000, June 2000, vol 2, pp. 134--141.
• 12: S. You, J. Hu, U. Neumann, and P. Fox, Urban Site Modeling From LiDAR, Second International Workshop on Computer Graphics and Geometric Modeling CGGM'2003, May 2003.
• 13: T. Kanade, R. Collins, A. Lipton, P. Burt and L. Wixson, "Advances in cooperative multi-sensor video surveillance", Proc. of DARPA Image Understanding Workshop, Vol. 1, pp. 3--24, 1998.
• 14: Ramesh Jain , Rangachar Kasturi , Brian G. Schunck, Machine vision, McGraw-Hill, Inc., New York, NY, 1995
• 15: U. Neumann and S. You, "Natural Feature Tracking for Augmented-Reality", IEEE Transactions on Multimedia. Vol. 1. No. 1, pp. 53--64, March 1999.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis

Additional Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.5 Computational Geometry and Object Modeling

General Terms:
Design, Security

Keywords:
augmented reality, object detection and tracking, video surveillance