Enabling multimedia using resource-constrained video processing techniques: A node-centric perspective

ABSTRACT

Successful proliferation of multimedia-enabled devices and advances in very large-scale integration (VLSI) technology has spawned new research efforts in migrating video processing applications onto ever smaller and more inexpensive devices. This article focuses on the technical challenges associated with that migration.

Due to limitations in size, battery lifetime, and, ultimately, cost, mapping complex video applications onto resource-constrained systems is a very challenging proposition. To this end, we first consider a technique, region-of-interest (ROI) processing, of defining a window within a video frame and only operating on the data inside that window, ignoring the rest of the frame. By using this lossy technique, the processing requirements can be reduced by roughly 80% while the error introduced in the quality of the results is roughly 10%. The other technique is adaptive data partitioning (ADP) combined with a content-based power management algorithm. By distributing video processing among multiple processors and shutting them down when they are not needed, the energy consumed per processor can be reduced by 60% without sacrificing the performance of the underlying video-based application.

Taken together, these novel techniques enable ambient multimedia systems and maintain the needed overall efficiency in video processing.

REFERENCES

• 1: Aarts, E. 2004. Ambient intelligence: A multimedia perspective. IEEE Multimedia Mag. 11, 1(Jan.), 12--19.
• 2: Altilar, D. and Paker, Y. 2001. Minimum overhead data partitioning algorithms for parallel video processing. In Proceedings of the International Conference on Domain Decomposition Methods.
• 3: Atmel. 2007. AT91 arm-based microcontrollers. http://www.atmel.com/products/AT91/.
• 4: Bradley, A. and Stentiford, F. 2002. JPEG 2000 and region of interest coding. Digit. Image Comput. Tech. Appl.
• 5: Andrea Cavallaro , Olivier Steiger , Touradj Ebrahimi, Multiple video object tracking in complex scenes, Proceedings of the tenth ACM international conference on Multimedia, December 01-06, 2002, Juan-les-Pins, France [doi>10.1145/641007.641115]
• 6: Chen, K. and Hu, C. 1998. Performance and VDD scaling in deep submicrometer CMOS. IEEE J. Solid-State Circ. 33, 10, 1586--1589.
• 7: Ellis, T., Black, J., Xu, M., and Makris, D. 2005. A distributed multicamera surveillance system. In Ambient Intelligence---A Novel Paradigm, Springer, New York, 107--138.
• 8: Ferscha, A., Vogl, S., and Wolfgang, B. 2002. Ubiquitous context sensing in wireless environments. In Proceedings of the 4th Austrian-Hungarian Workshop on Distributed and Parallel Systems (DAPSYS).
• 9: Hamamoto, T., Nagao, S., and Aizawa, K. 2002. Real-time objects tracking by using smart image sensor and FPGA. In Proceedings of the IEEE International Conference on Image Processing.
• 10: K. Hariharakrishnan , D. Schonfeld , P. Raffy , F. Yassa, Object tracking using adaptive block matching, Proceedings of the 2003 International Conference on Multimedia and Expo - Volume 3 (ICME '03), p.65-68, July 06-09, 2003
• 11: Hu, X., Ogras, U. Y., Zamora, N. H., and Marculescu, R. 2004. Data partitioning techniques for pervasive multimedia platforms. In Proceedings of the IEEE International Conference on Multimedia and Expo.
• 12: Huang, Y., Hsieh, B., Chien, S., and Chen, L. 2002. Simple and effective algorithm for automatic tracking of a single object using a pan-tilt-zoom camera. In Proceedings of the IEEE International Conference on Multimedia and Expo.
• 13: Cheolwhan Lee , Yuan Wang , Tao Yang, Static Global Scheduling for Optimal Computer Vision and Image Processing Operations on Distributed-Memory Multiprocessor, University of California at Santa Barbara, Santa Barbara, CA, 1994
• 14: Omnivision. 2007. OV7141 CMOS VGA. CameraChip. http://www.ovt.com/pdfs/pb_7141_7640.pdf.
• 15: Rabaey, J., Chandrakasan, A., and Nikolic', B. 2003. Digital Integrated Circuits, 2nd ed. Prentice Hall, Upper Saddle River, NJ, 213--231.
• 16: Kafil M. Razeeb , Stephen Bellis , Brendan O'Flynn , John Barton , Kieran Delaney , Cian O'Mathuna, A hybrid network of autonomous sensor nodes, Proceedings of the 2nd European Union symposium on Ambient intelligence, November 08-10, 2004, Eindhoven, Netherlands [doi>10.1145/1031419.1031436]
• 17: Senior, A., Hampapur, A., Brown, L., Tian, Y.-L., Shu, C.-F., and Pankanti, S. 2005. Distributed active multicamera networks. In Ambient Intelligence---A Novel Paradigm. Springer, New York, 89--105.
• 18: M. Ibrahim Sezan , Reginald L. Lagendijk, Motion Analysis and Image Sequence Processing (The International Series in Engineering and Computer Science), Springer-Verlag New York, Inc., Secaucus, NJ, 1993
• 19: Xinding Sun , Jonathan Foote , Don Kimber , B. S. Manjunath, Panoramic video capturing and compressed domain virtual camera control, Proceedings of the ninth ACM international conference on Multimedia, September 30-October 05, 2001, Ottawa, Canada [doi>10.1145/500141.500191]
• 20: Texas Instruments. 2007. TMS320VC5510. http://focus.ti.com/docs/prod/folders/print/tms320vc 5510a.html.
• 21: Wayne Wolf , Burak Ozer , Tiehan Lv, Smart Cameras as Embedded Systems, Computer, v.35 n.9, p.48-53, September 2002 [doi>10.1109/MC.2002.1033027]
• 22: Zamora, N. H., Hu, X., Ogras, U. Y., and Marculescu, R. 2004. Resource-aware video processing techniques for ambient multimedia systems. In Proceedings of the IEEE International Conference on Multimedia and Expo.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.1 Digitization and Image Capture
          Subjects: Sampling

Additional Classification:
  D. Software
  D.1 PROGRAMMING TECHNIQUES
      D.1.3 Concurrent Programming
          Subjects: Parallel programming

  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.6 Segmentation
          Subjects: Region growing, partitioning

Keywords:
Region-of-interest (ROI), data partitioning, lossy and lossless video processing, real-time video processing