Voice over IP performance monitoring

ABSTRACT

We describe a method for monitoring Voice over IP (VoIP) applications based upon a reduction of the ITU-T's E-Model to transport level, measurable quantities. In the process, 1) we identify the relevant transport level quantities, 2) we discuss the tradeoffs between placing the monitors within the VoIP gateways versus placement of the monitors within the transport path, and 3) we identify several areas where further work and consensus within the industry are required. We discover that the relevant transport level quantities are the delay, network packet loss and the decoder's de-jitter buffer packet loss. We find that an in-path monitor requires the definition of a reference de-jitter buffer implementation to estimate voice quality based upon observed transport measurements. Finally, we suggest that more studies are required, which evaluate the quality of various VoIP codecs in the presence of representative packet loss patterns.

REFERENCES

• 1: Jacobson, V., "VAT manual pages", Lawrence Berkeley Laboratory, Feb. 1992; www.nrg.ee.lbl.gov/vat/.
• 2: Hardman, V., Kinstein, P., Sasse, A. and I. Kouvelas, "Robust Audio Tool (RAT)", www-mice.cs.ucl.ac.uk/multimedia/software/rat/index.htm.
• 3: ITU-T Recommendation G.107, "The E-Model, a computational model for use in transmission planning", December 1998.
• 4: "The Cisco-Voice-Dial-Control-MIB", CiscoSystems, Inc., ftp://ftp.cisco.com/pub/mibs/v2.
• 5: "The Telchemy monitoring agent", Telchemy, Inc., www.telchemy.com.
• 6: ITU-T, Recommendation P.861: "Objective quality measurement of telephone-band (300-3400 Hz) speech codes," August 1996.
• 7: Dr. John G. Beerends, Mr. Antony Rix, Dr. Andreis P. Hekstra and Dr. Mike Hollier, "Proposed Draft Recommendation P.86x: Perceptual Evaluation of Speech Quality (PESQ), an Objective Method for End-to-end Speech Quality Assessment of Narrowband Telephone Networks and Speech Codecs" ITU-T Delayed Contribution 12-D.140-E, May 2000.
• 8: ITU-T Recommendation G.113, "General Characteristics of General Telephone Connections and Telephone Circuits --- Transmission Impairments", February 1996.
• 9: Perkins, M, Dvorak, C., Leisch, B., and J. A. Zebarth, "Speech Transmission Performance Planning in Hybrid IP/SCN Networks", IEEE Communications Magazine, July 1999.
• 10: Private communications with Mark Perkins of AT&T.
• 11: Annex C of Reference {3} contains code to compute the R-factor and obtain the expressions shown in Equation (3) and Table 2.
• 12: Hardman, V. J., Sasse, M. A., Watson, A. and M. Handley, "Reliable Audio for use over the Internet", In Proceedings of INET95 (Honolulu, Oahu, Hawaii, Sept. 1995); info.isoc.org/HMP/PAPER/070/html/paper.html.
• 13: Vicky Hardman , Martina Angela Sasse , Isidor Kouvelas, Successful multiparty audio communication over the Internet, Communications of the ACM, v.41 n.5, p.74-80, May 1998 [doi>10.1145/274946.274959]
• 14: ITU-T Recommendation G.729a, "Coding of Speech at 8 kbit/s using Conjugate-structure Algebraic-Code-Excited Linear-Prediction (SC-ACELP)", March 1996.
• 15: ITU-T Recommendation G.711, "Pulse Code modulation (PCM) of voice frequencies", November 1988.
• 16: ANSI T1.521-1999, "Packet Loss Concealment for use with ITU-T Recommendation G.711", December 1999.
• 17: Bolot, J.-C., "Characterizing end-to-end packet delay and loss in the Internet", Journal of High Speed Networks, vol. 2, pp 305-323, 1993.
• 18: M. S. Borella, D. Swider, S. Uludag, G. B. Brewster, "Analysis of end-to-end internet packet loss: Dependence and asymmetry", Technical Report AT031798, 3COM Advanced technologies, March 1998.
• 19: R. Ramjee, J. Kurose, D. Towsley, H. Schulzrinne, "Adaptive Playout Mechanisms for packetized Audio Applications in Wide Area Networks", Proc. IEEE Infocom, 1994.
• 20: Sue B. Moon , Jim Kurose , Don Towsley, Packet audio playout delay adjustment: performance bounds and algorithms, Multimedia Systems, v.6 n.1, p.17-28, Jan. 1998 [doi>10.1007/s005300050073]
• 21: DTS/TIPHON 5008 VO.2.5, "Quality of Service (QOS) measurement methodologies", July 2000 and private communications with A. Clark.
• 22: Feller, W., "Introduction to Probability Theory, Vol.1", John Wiley & Sons, New York, 1968.
• 23: Kouvelas, I. And V. J. Hardman, "Overcoming Workstation Scheduling Problems in a Real Time Audio Tool", Usenix Conference, June 1997.
• 24: The RMONMIB Home Page at http://www.ietf.org/html.charters/rmonmib-charter.html/
• 25: The UCD SNMP Project, see http://ucd-snmp.ucdavis.edu/, the University of California, Davis, Calif.
• 26: The Cisco rttMonMIB, http://www.cisco.com/public/mibs/v2/CISCO-RTTMON-120 5 T.my, Cisco Systems, San Jose, Calif., USA.
• 27: Redhat Linux 6.1, see http://www.redhat.com/, the Red Hat, Inc., Research Triangle Park, North Carolina, USA.