Predictive perceptual compression for real time video communication

ABSTRACT

Approximately 2 degrees in our 140 degree vision span has sharp vision. Many researchers have been fascinated by the idea of eye-tracking integrated perceptual compression of an image or video, yet any practical system has yet to emerge. The unique challenge presented by real time perceptual video streaming is how to handle the fast nature of the human eye and provide its integration with computationally intensive video transcoding scheme. The delay introduced by video transmission in the network presents a difficulty. This delay creates a problem when we try to use information about eye movements for perceptual encoding. In this paper we discuss a new approach to the eye-tracker based video compression. Rather than relying on the point of gaze, this novel scheme tracks a vicinity of interest and offers a prediction mechanism for eye movements. The described system compensates the interim eye movements between the sampling and actual coding. The proposed scheme can be applied to a large variety of today's video compression standards. We have developed an eye gaze-aware MPEG-2 transcoder that can perceptually re-encode a live video stream in real time. The experiments we have conducted illustrate the substantial impact this integrated prediction method has on perceptual video compression and bit-rate reduction.

REFERENCES

• 1: Daly, S. J., Matthews, K., Ribas-Corbera, J. Visual eccentricity models in face-based video compression. In Human Vision and Electronic Imaging IV, May 1995, SPIE.
• 2: Duchowski, A. T. Acuity-Matching Resolution Degradation Through Wavelet Coefficient Scaling. IEEE Transactions on Image Processing 9, 8. August 2000.
• 3: Geisler, W. S., Perry, J. S. Real-time Foveated Multiresolution System for Low-bandwidth Video Communication. In Human Vision and Electronic Imaging III, July 1998, SPIE.
• 4: Irwin, D. E. Visual Memory Within and Across Fixations. In Eye movements and Visual Cognition: Scene Preparation And Reading, K. Rayner, Ed. Springer-Verlag, New-York, NY,1992, pp. 146--165. Springer Series in Neuropsychology.
• 5: Khan, J., Gu, Q., and Zaghal, R. Symbiotic Video Streaming by Transport Feedback based Quality rate Selection In Proceedings of the 12th IEEE International Packet Video Workshop 2002, Pittsburg, PA, April 2002,
• 6: Khan.J., Yun D. Multi Resolution Perceptual Encoding for Interactive Image Sharing in Remote Tele-Diagnostics, Manufacturing Agility and Hybrid Automation -I. In Proceedings of the International Conference on Human Aspects of Advanced Manufacturing: Agility & Hybrid Automation, HAAMAHA'96, Hawaii, Aug. 1996, pp183--187.
• 7: Komogortsev, O., Khan, J., Video Set for Predictive Perceptual Compression Test. At www.cs.kent.edu/~okomogor/ACM04VideoSet.htm.
• 8: Kuyel, T., Geisler, W. S., Ghosh, J. Retinally reconstructed images (RRIs): digital images having a resolution match with the human eye. In Human Vision and Electronic Imaging III, July 1998, SPIE .
• 9: Lee, S., Pattichis, M., Bovok, A. Foveated Video Compression with Optimal Rate Control. In IEEE Transaction of Image Processing, V. 10, n.7, July 2001, pp-977--992.
• 10: Lester C. Loschky , George W. McConkie, User performance with gaze contingent multiresolutional displays, Proceedings of the 2000 symposium on Eye tracking research & applications, p.97-103, November 06-08, 2000, Palm Beach Gardens, Florida, United States [doi>10.1145/355017.355032]
• 11: Niu, E. L. Gaze-based video compression using wavelets. M.S. Thesis. University of Illinois at Urbana-Champaign. The Graduate College. August 1995.
• 12: Wang, Z., Lu, L., and Bovik, A. Rate scalable video coding using a foveation-based human visual system model. IEEE International Conference on Acoustics, Speech, & Signal Processing, May 2001.
• 13: Westen, S. J., Lagendijk, R., Biemond, J. Spatio-temporal model of human vision for digital video compression. In Human Vision and Electronic Imaging II, June 1997, SPIE.
• 14: Yoon, S., Ratakonda, K., Ahuja, N. Region-Based Video Coding Using A Multiscale Image Segmentation. In Proceedings of 1997 International Conference on Image Processing (ICIP '97).
• 15: Kortum, P., Geisler, W. S., Implementation of a Foveated Image Coding System for Image Bandwidth Reduction. In Human Vision and Electronic Imaging, April 1996, SPIE.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.4 INFORMATION SYSTEMS APPLICATIONS
      H.4.3 Communications Applications
          Subjects: Computer conferencing, teleconferencing, and videoconferencing

Additional Classification:
  J. Computer Applications
  J.7 COMPUTERS IN OTHER SYSTEMS
      Subjects: Real time; Process control

General Terms:
Algorithms, Design, Experimentation, Performance

Keywords:
perceptual compression, video transcoding