Quantifying Skype user satisfaction

ABSTRACT

The success of Skype has inspired a generation of peer-to-peer-based solutions for satisfactory real-time multimedia services over the Internet. However, fundamental questions, such as whether VoIP services like Skype are good enough in terms of user satisfaction,have not been formally addressed. One of the major challenges lies in the lack of an easily accessible and objective index to quantify the degree of user satisfaction.In this work, we propose a model, geared to Skype, but generalizable to other VoIP services, to quantify VoIP user satisfaction based on a rigorous analysis of the call duration from actual Skype traces. The User Satisfaction Index (USI) derived from the model is unique in that 1) it is composed by objective source-and network-level metrics, such as the bit rate, bit rate jitter, and round-trip time, 2) unlike speech quality measures based on voice signals, such as the PESQ model standardized by ITU-T, the metrics are easily accessible and computable for real-time adaptation, and 3) the model development only requires network measurements, i.e., no user surveys or voice signals are necessary. Our model is validated by an independent set of metrics that quantifies the degree of user interaction from the actual traces.

REFERENCES

• 1: S. A. Baset and H. Schulzrinne. An analysis of the Skype peer-to-peer internet telephony protocol. In Proceedings of IEEE INFOCOM '06, Barcelona, Spain, Apr. 2006.
• 2: T. Berson. Skype security evaluation. ALR-2005-031, Anagram Laboratorie, 2005.
• 3: D. R. Cox and D. Oakes. Analysis of Survival Data. Chapman & Hall/CRC, June 1984.
• 4: D. R. Cox and E. J. Snell. A general definition of residuals (with discussion). Journal of the Royal Statistical Society, B 30:248--275, 1968.
• 5: I. Daubechies. The wavelet transform, time-frequency localization and signal analysis. IEEE Transactions on Information Theory, 36(5):961--1005, Sept. 1990.
• 6: D. L. Donoho. De-noising by soft-thresholding. IEEE Transactions on Information Theory, 41(3):613--627, May 1995.
• 7: Frank E. Harrell, Jr., Regression Modeling Strategies, Springer-Verlag New York, Inc., Secaucus, NJ, 2006
• 8: D. P. Harrington and T. R. Fleming. A class of rank test procedures for censored survival data. Biometrika, 69:553--566, 1982.
• 9: ITU-T Recommendation P.862. Perceptual evaluation of speech quality (PESQ), an objective method for end-to-end speech quality assessment of narrow-band telephone networks and speech codecs, Feb 2001.
• 10: K. Lam, O. Au, C. Chan, K. Hui, and S. Lau. Objective speech quality measure for cellular phone. In Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, volume 1, pages 487--490, 1996.
• 11: A. Rix, J. Beerends, M. Hollier, and A. Hekstra. Perceptual evaluation of speech quality (PESQ)-a new method for speech quality assessment of telephone networks and codecs. In proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, volume 2, pages 73--76, 2001.
• 12: K. Suh, D. R. Figueiredo, J. Kurose, and D. Towsley. Characterizing and detecting relayed traffic: A case study using Skype. In Proceedings of IEEE INFOCOM '06, Barcelona, Spain, Apr. 2006.
• 13: T. M. Therneau and P. M. Grambsch. Modeling Survival Data: Extending the Cox Model. Springer, 1st edition, August 2001.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.4 INFORMATION SYSTEMS APPLICATIONS
      H.4.3 Communications Applications
          Subjects: Computer conferencing, teleconferencing, and videoconferencing

Additional Classification:
  G. Mathematics of Computing
  G.3 PROBABILITY AND STATISTICS
      Subjects: Survival analysis

  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human factors

General Terms:
Human Factors, Measurement, Performance

Keywords:
VoIP, human perception, internet measurement, quality of service, survival analysis, wavelet denoising