I tube, you tube, everybody tubes: analyzing the world's largest user generated content video system

ABSTRACT

User Generated Content (UGC) is re-shaping the way people watch video and TV, with millions of video producers and consumers. In particular, UGC sites are creating new viewing patterns and social interactions, empowering users to be more creative, and developing new business opportunities. To better understand the impact of UGC systems, we have analyzed YouTube, the world's largest UGC VoD system. Based on a large amount of data collected, we provide an in-depth study of YouTube and other similar UGC systems. In particular, we study the popularity life-cycle of videos, the intrinsic statistical properties of requests and their relationship with video age, and the level of content aliasing or of illegal content in the system. We also provide insights on the potential for more efficient UGC VoD systems (e.g. utilizing P2P techniques or making better use of caching). Finally, we discuss the opportunities to leverage the latent demand for niche videos that are not reached today due to information filtering effects or other system scarcity distortions. Overall, we believe that the results presented in this paper are crucial in understanding UGC systems and can provide valuable information to ISPs, site administrators, and content owners with major commercial and technical implications.

REFERENCES

• 1: Daum UCC. http://ucc.daum.net.
• 2: Imdb statistics. http://www.imdb.com/database_statistics.
• 3: Lovefilm. http://www.lovefilm.com.
• 4: Netflix prize. http://www.netflixprize.com.
• 5: Yahoo! Movies. http://movies.yahoo.com.
• 6: YouTube. http://www.youtube.com.
• 7: Surveys: Internet Traffic Touched by YouTube, January 2006. http://www.lightreading.com/document.asp?doc_id=115816.
• 8: L. Amaral, A. Scala, M. Barthélémy, and H. E. Stanley. Classes of Small-World Networks. In Proc. Natl. Acad. Sci. USA, 2000.
• 9: C. Anderson. A Problem With the LongTail. http://www.longtail.com/scifoo.ppt.
• 10: Chris Anderson, The Long Tail: Why the Future of Business Is Selling Less of More, Hyperion, 2006
• 11: E. Auchard. Participation on Web 2.0 Sites Remains Weak, April 2007. http://www.reuters.com/article/internetNews/idUSN1743638820070418.
• 12: A.-L. Barabási and R. Albert. Emergence of Scaling in Random Networks. Science, 286:509--512, 1999.
• 13: S. Bausch and L. Han. YouTube U.S. Web Traffic Grows 75 Percent Week over Week, July 2006. Neilsen/Netratings, http://www.nielsen-netratings.com/pr/pr_060721_2.pdf.
• 14: B. Cheng, X. Liu, Z. Zhang, and H. Jin. A Measurement Study of a Peer-to-Peer Video-on-Demand System. In Proc. of IPTPS, 2007.
• 15: Junghoo Cho , Sourashis Roy, Impact of search engines on page popularity, Proceedings of the 13th international conference on World Wide Web, May 17-20, 2004, New York, NY, USA [doi>10.1145/988672.988676]
• 16: Cristiano P. Costa , Italo S. Cunha , Alex Borges , Claudiney V. Ramos , Marcus M. Rocha , Jussara M. Almeida , Berthier Ribeiro-Neto, Analyzing client interactivity in streaming media, Proceedings of the 13th international conference on World Wide Web, May 17-20, 2004, New York, NY, USA [doi>10.1145/988672.988744]
• 17: Mark E. Crovella , Azer Bestavros, Self-similarity in World Wide Web traffic: evidence and possible causes, IEEE/ACM Transactions on Networking (TON), v.5 n.6, p.835-846, Dec. 1997 [doi>10.1109/90.650143]
• 18: T. Do, K. A. Hua, and M. Tantaoui. P2VoD: Providing Fault Tolerant Video-on-Demand Streaming in Peer-to-Peer Environment. Proc. of IEEE ICC, 2004.
• 19: Allen B. Downey, The Structural Cause of File Size Distributions, Proceedings of the Ninth International Symposium in Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS'01), p.361, August 15-18, 2001
• 20: T. Fenner, M. Levene, and G. Loizou. A Stochastic Evolutionary Model Exhibiting Power-Law Behaviour with an Exponential Cutoff. Physica, (13), 2005.
• 21: S. Fortunato, A. Flammini, F. Menczer, and A. Vespignani. Topical Interests and the Mitigation of Search Engine Bias. In Proc. Natl. Acad. Sci. USA, 2006.
• 22: Christos Gkantsidis , Thomas Karagiannis , Milan VojnoviC, Planet scale software updates, Proceedings of the 2006 conference on Applications, technologies, architectures, and protocols for computer communications, September 11-15, 2006, Pisa, Italy
• 23: L. Gomes. Will all of us get our 15 minutes on a youtube video?, The Wall Street Journal Online, August 2006.
• 24: Carsten Griwodz , Michael Bär , Lars C. Wolf, Long-term movie popularity models in video-on-demand systems: or the life of an on-demand movie, Proceedings of the fifth ACM international conference on Multimedia, p.349-357, November 09-13, 1997, Seattle, Washington, United States [doi>10.1145/266180.266386]
• 25: Siddhartha Annapureddy , Saikat Guha , Christos Gkantsidis , Dinan Gunawardena , Pablo Rodriguez Rodriguez, Is high-quality vod feasible using P2P swarming?, Proceedings of the 16th international conference on World Wide Web, May 08-12, 2007, Banff, Alberta, Canada [doi>10.1145/1242572.1242694]
• 26: Krishna P. Gummadi , Richard J. Dunn , Stefan Saroiu , Steven D. Gribble , Henry M. Levy , John Zahorjan, Measurement, modeling, and analysis of a peer-to-peer file-sharing workload, Proceedings of the nineteenth ACM symposium on Operating systems principles, October 19-22, 2003, Bolton Landing, NY, USA
• 27: Yang Guo , Kyoungwon Suh , Jim Kurose , Don Towsley, P2Cast: peer-to-peer patching scheme for VoD service, Proceedings of the 12th international conference on World Wide Web, May 20-24, 2003, Budapest, Hungary [doi>10.1145/775152.775195]
• 28: B. Holt, H. R. Lynn, and M. Sowers. Analysis of Copyrighted Videos on YouTube.com. http://www.vidmeter.com/i/vidmeter_copyright_report.pdf.
• 29: C. Huang, J. Li, and K. Ross. Peer-Assisted VoD: Making Internet Video Distribution Cheap. In Proc. of IPTPS, 2007.
• 30: Y. Ijiri and H. Simon. Skew Distributions and the Size of Business Firms. North Holland, Amsterdam, 1977.
• 31: D. A. L. Li, J. Doyle, and W. Willinger. Towards a Theory of Scale-Free Graphs: Definition, Properties, and Implications. Internet Mathematics, 2(4), 2006.
• 32: E. Limpert, W. A. Stahel, and M. Abbt. Log-normal Distributions across the Sciences: Keys and Clues. BioScience, 51(5):341, 2001.
• 33: N. Magharei and R. Rejaie. PRIME: Peer-to-Peer Receiver-drIven MEsh-based Streaming. In Proc. of IEEE INFOCOM, 2007.
• 34: N. Miller. Manifesto for a New Age. Wired Magazine, March 2007.
• 35: M. Mitzenmacher. A Brief History of Generative Models for Power Law and Lognormal Distributions. Internet Mathematics, 1(2):226--251, 2004.
• 36: S. Mossa, M. Barthélémy, H. E. Stanley, and L. A. N. Amaral1. Truncation of Power Law Behavior in "Scale-Free" Network Models due to Information Filtering. Phys. Rev. Lett., (13), 2002.
• 37: M. E. J. Newman. Power laws, Pareto distributions and Zipf 's law. Contemporary Physics, 46:323, 2005.
• 38: V. M. W. Gong, Y. Liu and D. Towsley. On the Tails of Web File Size Distributions. In Proc. of 39th Allerton Conference on Communication, Control, and Computing, 2001.
• 39: Hongliang Yu , Dongdong Zheng , Ben Y. Zhao , Weimin Zheng, Understanding user behavior in large-scale video-on-demand systems, Proceedings of the 1st ACM SIGOPS/EuroSys European Conference on Computer Systems 2006, April 18-21, 2006, Leuven, Belgium
• 40: G. U. Yule. A Mathematical Theory of Evolution, Based on the Conclusions of Dr. J. C. Willis, F. R. S. Royal Society of London Philosophical Transactions Series B, 213:21--87, 1925.

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.0 General

General Terms:
Design, Measurement

Keywords:
caching, content aliasing, long tail, p2p, popularity analysis, power-law, user generated content, vod