Real-time monitoring of video quality in IP networks

ABSTRACT

This paper investigates the problem of assessing the quality of video transmitted over IP networks. Our goal is to develop a methodology that is both reasonably accurate and simple enough to support the large-scale deployments that the increasing use of video over IP are likely to demand. For that purpose, we focus on developing an approach that is capable of mapping network statistics, e.g., packet losses, available from simple measurements, to the quality of video sequences reconstructed by receivers. A first step in that direction is a loss-distortion model that accounts for the impact of network losses on video quality, as a function of application-specific parameters such as video codec, loss recovery technique, coded bit rate, packetization, video characteristics, etc. The model, although accurate, is poorly suited to large-scale, on-line monitoring, because of its dependency on parameters that are difficult to estimate in real-time. As a result, we introduce a "relative quality" metric (rPSNR) that bypasses this problem by measuring video quality against a quality benchmark that the network is expected to provide. The approach offers a lightweight video quality monitoring solution that is suitable for large-scale deployments. We assess its feasibility and accuracy through extensive simulations and experiments.

REFERENCES

• 1: W. B. Norton, "Internet video: The next wave of massive disruption to US peering ecosystem," presented at the Asia Pacific Regional Internet Conf. Operational Technologies (APRICOT), Bali, Indonesia, Feb. 2007.
• 2: Dmitri Loguinov , Hayder Radha, Measurement study of low-bitrate internet video streaming, Proceedings of the 1st ACM SIGCOMM Workshop on Internet Measurement, November 01-02, 2001, San Francisco, California, USA [doi>10.1145/505202.505238]
• 3: D. Loguinov and H. Radha, "End-to-end Internet video traffic dynamics: Statistical study and analysis," presented at the IEEE INFOCOM, New York, Jun. 2002.
• 4: Yubing Wang , Mark Claypool , Zheng Zuo, An empirical study of realvideo performance across the internet, Proceedings of the 1st ACM SIGCOMM Workshop on Internet Measurement, November 01-02, 2001, San Francisco, California, USA [doi>10.1145/505202.505239]
• 5: Mingzhe Li , Mark Claypool , Robert Kinicki, MediaPlayer™ versus RealPlayer™: a comparison of network turbulence, Proceedings of the 2nd ACM SIGCOMM Workshop on Internet measurment, November 06-08, 2002, Marseille, France [doi>10.1145/637201.637221]
• 6: F. Li, J. Chung, M. Li, H. Wu, M. Claypool, and R. Kinicki, "Application, network and link layer measurements of streaming video over a wireless campus network," presented at the 6th Passive and Active Network Measurement Workshop (PAM), Boston, MA, April 2005.
• 7: P. Frossard and O. Verscheure, "Joint source/FEC rate selection for quality-optimal MPEG-2 video delivery," IEEE Trans. Image Processi., vol. 10, no. 12, pp. 1815-1825, December 2001.
• 8: W. Jiang and H. Schulzrinne, "Modeling of packet loss and delay and their effect on real-time multimedia service quality," presented at the NOSSDAV, Chapel Hill, NC, Jun. 2000.
• 9: Y. J. Liang, J. G. Apostolopoulos, and B. Girod, "Analysis of packet loss for compressed video: Does burst-length matter?," presented at the ICASSP, Hong Kong, China, Apr. 2003.
• 10: A. R. Reibman , V. Vaishampayan, Quality monitoring for compressed video subjected to packet loss, Proceedings of the 2003 International Conference on Multimedia and Expo, p.17-20, July 06-09, 2003
• 11: A. R. Reibman and V. Vaishampayan, "Low complexity quality monitoring of MPEG-2 video in a network," presented at the IEEE ICIP, Middletown, NJ, Sep. 2003.
• 12: A. R. Reibman, V. Vaishampayan, and Y. Sermadevi, "Quality monitoring of video over a packet network," IEEE Trans. Multimedia, vol. 6, no. 2, pp. 327-334, Apr. 2004.
• 13: S. Kanumuri, P. C. Cosman, A. R. Reibman, and V. A. Vaishampayan, "Modeling packet-loss visibility in MPEG-2 video," IEEE Trans. Multimedia , vol. 8, no. 2, pp. 341-355, Apr. 2006.
• 14: Jill M. Boyce , Robert D. Gaglianello, Packet loss effects on MPEG video sent over the public Internet, Proceedings of the sixth ACM international conference on Multimedia, p.181-190, September 13-16, 1998, Bristol, United Kingdom [doi>10.1145/290747.290770]
• 15: L. Rizzo, Dummynet. [Online]. Available: http://www.info.iet.unipi.it/ luigi.
• 16: K. Stuhlmuller, N. Farber, M. Link, and B. Girod, "Analysis of video transmission over lossy channels," IEEE J. Sel. Areas Commun., vol. 1, no. 6, pp. 1012-1032, Jun. 2000.
• 17: Shu Tao , John Apostolopoulos , Roch Guérin, Real-time monitoring of video quality in IP networks, Proceedings of the international workshop on Network and operating systems support for digital audio and video, June 13-14, 2005, Stevenson, Washington, USA [doi>10.1145/1065983.1066013]
• 18: S. Tao, "Improving the Quality of Real-Time Applications Through Path Switching," Ph.D. dissertation, Dept. Electrical Syst. Eng., Univ. Pennsylvania, Philadelphia, Nov. 2005.
• 19: Shu Tao , Roch Guérin, Application-specific path switching: a case study for streaming video, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027553]
• 20: S. Tao and R. Guerin, "On-line estimation of Internet path performance: An application perspective," presented at the IEEE INFOCOM, Hong Kong, China, Mar. 2004.
• 21: A. C. Dalal and E. Perry, "A new architecture for measuring and assessing streaming media quality," presented at the 3rd Workshop Passive Active Measurements (PAM 2003), San Diego, CA, Apr. 2003.
• 22: Jacobus van der Merwe , Ramón Cáceres , Yang-hua Chu , Cormac Sreenan, mmdump: a tool for monitoring internet multimedia traffic, ACM SIGCOMM Computer Communication Review, v.30 n.5, October 2000 [doi>10.1145/505672.505678]
• 23: Oliver Verscheure , Pascal Frossard , Maber Hamdi, User-oriented QoS analysis in MPEG-2 video delivery, Real-Time Imaging, v.5 n.5, p.305-314, Oct. 1999 [doi>10.1006/rtim.1999.0175]
• 24: M. Dai, D. Loguinov, and H. Radha, "Rate-distortion modeling of scalable video coders," presented at the IEEE ICIP, Singapore, Oct. 2004.
• 25: C. Perkins, O. Hodson, and V. Hardman, "Survey of packet loss recovery techniques for streaming audio," IEEE Network, vol. 12, pp. 40-48, Sep. 1998.
• 26: N. Feamster and H. Balakrishnan, "Packet loss recovery for streaming video," presented at the 2th Int. Packet Video Workshop, Pittsburgh, PA, Apr. 2002.
• 27: A. Basso, G. Cash, and M. Civanlar, "Transmission of MPEG-2 streams over non-guaranteed quality of service networks," presented at the Picture Coding Symp., Berlin, Germany, Sep. 1997.
• 28: Injong Rhee, Error control techniques for interactive low-bit rate video transmission over the Internet, Proceedings of the ACM SIGCOMM '98 conference on Applications, technologies, architectures, and protocols for computer communication, p.290-301, August 31-September 04, 1998, Vancouver, British Columbia, Canada
• 29: Benjamin W. Wah , Xiao Su , Dong Lin, A Survey of Error-Concealment Schemes for Real-Time Audio and Video Transmissions over the Internet*, Proceedings of the 2000 International Conference on Microelectronic Systems Education, p.17, November 13-13, 2000
• 30: VQEG, Final Report on the Validation of Objective Models of Video Quality Assessment. Aug. 2003 [Online]. Available: http://www.vqeg. org/
• 31: D. Hoffman , G. Fernando , V. Goyal , M. Civanlar, RTP Payload Format for MPEG1/MPEG2 Video, RFC Editor, 1998
• 32: H. Schulzrinne, S. Casner, R. Frederick, and V. Jacobson, "RTP: A Transport protocol for real-time applications," RFC 1889, Jan. 1996.
• 33: libmpeg2. [Online]. Available: http://www.libmpeg2.sourceforge.net/
• 34: H.264/AVC Software Coordination. Reference Software (Version JM 8.6). [Online]. Available: http://www.iphome.hhi.de/suehring/tml/ index.htm/

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.3 Network Operations
          Subjects: Network monitoring

Additional Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.2 Network Protocols
          Subjects: Applications (SMTP, FTP, etc.)
          Nouns: IP
  C.4 PERFORMANCE OF SYSTEMS
      Subjects: Performance attributes

  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Video (e.g., tape, disk, DVI)

General Terms:
Design, Management, Performance

Keywords:
IP networks, PSNR, relative video quality, video quality