Delay tolerant bulk data transfers on the internet

ABSTRACT

Many emerging scientific and industrial applications require transferring multiple Tbytes of data on a daily basis. Examples include pushing scientific data from particle accelerators/colliders to laboratories around the world, synchronizing data-centers across continents, and replicating collections of high definition videos from events taking place at different time-zones. A key property of all above applications is their ability to tolerate delivery delays ranging from a few hours to a few days. Such Delay Tolerant Bulk (DTB) data are currently being serviced mostly by the postal system using hard drives and DVDs, or by expensive dedicated networks. In this work we propose transmitting such data through commercial ISPs by taking advantage of already-paid-for off-peak bandwidth resulting from diurnal traffic patterns and percentile pricing. We show that between sender-receiver pairs with small time-zone difference, simple source scheduling policies are able to take advantage of most of the existing off-peak capacity. When the time-zone difference increases, taking advantage of the full capacity requires performing store-and-forward through intermediate storage nodes. We present an extensive evaluation of the two options based on traffic data from 200+ links of a large transit provider with PoPs at three continents. Our results indicate that there exists huge potential for performing multi Tbyte transfers on a daily basis at little or no additional cost.

REFERENCES

• 1: Vinay Aggarwal , Anja Feldmann , Christian Scheideler, Can ISPS and P2P users cooperate for improved performance?, ACM SIGCOMM Computer Communication Review, v.37 n.3, July 2007 [doi>10.1145/1273445.1273449]
• 2: Eli Brosh , Salman Abdul Baset , Dan Rubenstein , Henning Schulzrinne, The delay-friendliness of TCP, Proceedings of the 2008 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, June 02-06, 2008, Annapolis, MD, USA
• 3: Joe Chabarek, Joel Sommers, Paul Barford, Cristian Estan, David Tsiang, and Steve Wright. Power Awareness in Network Design and Routing. In Proceedings of IEEE INFOCOM'08.
• 4: Augustin Chaintreau , Abderrahmen Mtibaa , Laurent Massoulie , Christophe Diot, The diameter of opportunistic mobile networks, Proceedings of the 2007 ACM CoNEXT conference, December 10-13, 2007, New York, New York [doi>10.1145/1364654.1364670]
• 5: Costas Courcoubetis , Richard Weber, Pricing Communication Networks: Economics, Technology and Modelling (Wiley Interscience Series in Systems and Optimization), John Wiley & Sons, 2003
• 6: Amogh Dhamdhere and Constantinos Dovrolis. ISP and Egress Path Selection for Multihomed Networks. In Proceedings of IEEE INFOCOM'06.
• 7: Vijay Erramilli , Mark Crovella , Augustin Chaintreau , Christophe Diot, Delegation forwarding, Proceedings of the 9th ACM international symposium on Mobile ad hoc networking and computing, May 26-30, 2008, Hong Kong, Hong Kong, China [doi>10.1145/1374618.1374653]
• 8: Vijay Erramilli , Augustin Chaintreau , Mark Crovella , Christophe Diot, Diversity of forwarding paths in pocket switched networks, Proceedings of the 7th ACM SIGCOMM conference on Internet measurement, October 24-26, 2007, San Diego, California, USA [doi>10.1145/1298306.1298330]
• 9: David K. Goldenberg , Lili Qiuy , Haiyong Xie , Yang Richard Yang , Yin Zhang, Optimizing cost and performance for multihoming, Proceedings of the 2004 conference on Applications, technologies, architectures, and protocols for computer communications, August 30-September 03, 2004, Portland, Oregon, USA
• 10: Sushant Jain , Kevin Fall , Rabin Patra, Routing in a delay tolerant network, Proceedings of the 2004 conference on Applications, technologies, architectures, and protocols for computer communications, August 30-September 03, 2004, Portland, Oregon, USA
• 11: Anukool Lakhina , Konstantina Papagiannaki , Mark Crovella , Christophe Diot , Eric D. Kolaczyk , Nina Taft, Structural analysis of network traffic flows, Proceedings of the joint international conference on Measurement and modeling of computer systems, June 10-14, 2004, New York, NY, USA
• 12: Matthew Roughan , Albert Greenberg , Charles Kalmanek , Michael Rumsewicz , Jennifer Yates , Yin Zhang, Experience in measuring backbone traffic variability: models, metrics, measurements and meaning, Proceedings of the 2nd ACM SIGCOMM Workshop on Internet measurment, November 06-08, 2002, Marseille, France [doi>10.1145/637201.637213]
• 13: James D. Salehi , Shi-Li Zhang , Jim Kurose , Don Towsley, Supporting stored video: reducing rate variability and end-to-end resource requirements through optimal smoothing, IEEE/ACM Transactions on Networking (TON), v.6 n.4, p.397-410, Aug. 1998 [doi>10.1109/90.720873]
• 14: S. Shalunov and B. Teitelbaum. Qbone Scavenger Service (QBSS) Definition. Internet2 technical report, March 2001.
• 15: R. Sherwood, R. Braud, and B. Bhattacharjee. Slurpie: A Cooperative Bulk Data Transfer Protocol. In Proceedings of IEEE INFOCOM'04.
• 16: Matti Siekkinen , Guillaume Urvoy-Keller , Ernst W. Biersack , Denis Collange, A root cause analysis toolkit for TCP, Computer Networks: The International Journal of Computer and Telecommunications Networking, v.52 n.9, p.1846-1858, June, 2008 [doi>10.1016/j.comnet.2008.03.005]
• 17: Hao Wang, Haiyong Xie, Lili Qiu, Abraham Silberschatz, and Yang Richard Yang. Optimal ISP Subscription for Internet Multihoming: Algorithm Design and Implication Analysis. In Proceedings of IEEE INFOCOM'05.
• 18: Randolph Y. Wang , Sumeet Sobti , Nitin Garg , Elisha Ziskind , Junwen Lai , Arvind Krishnamurthy, Turning the postal system into a generic digital communication mechanism, Proceedings of the 2004 conference on Applications, technologies, architectures, and protocols for computer communications, August 30-September 03, 2004, Portland, Oregon, USA
• 19: Haiyong Xie , Y. Richard Yang , Arvind Krishnamurthy , Yanbin Grace Liu , Abraham Silberschatz, P4p: provider portal for applications, Proceedings of the ACM SIGCOMM 2008 conference on Data communication, August 17-22, 2008, Seattle, WA, USA

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.1 Network Architecture and Design
          Subjects: Store and forward networks

Additional Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.2 Network Protocols
          Subjects: Routing protocols

General Terms:
Economics, Measurement, Performance

Keywords:
95-percentile pricing, content distribution, delay tolerant networks