On the impact of research network based testbeds on wide-area experiments

ABSTRACT

An important stage of wide-area systems and networking research is to prototype a system to understand its performance when deployed in the real Internet. A key requirement of prototyping is that results obtained from the prototype experiments be representative of the behavior if the system were deployed over nodes connected to commercial ISPs. Recently, distributed testbeds such as PlanetLab and RON have become increasingly popular for performing wide-area experimentation. However, such testbeds typically consist of a significant fraction of nodes with connectivity to research and education networks which potentially hinder their usability in prototyping systems.In this paper, we investigate the impact of testbeds with connectivity to research and education networks on the applications and network services so that such testbeds can be leveraged for evaluation and prototyping. Specifically, we investigate when the representativeness of wide-area experiments deployed on such testbeds is affected by studying the routing paths that applications use over such testbeds. We then investigate how the representativeness of wide-area experiments is affected by studying the performance properties of such paths. We further measure the impact of using such testbeds on application performance via application case studies. Finally, we propose a technique that uses the currently available testbeds but reduces their bias by exposing applications evaluated to network conditions more reflective of the conditions in the commercial Internet.

REFERENCES

• 1: David Andersen , Hari Balakrishnan , Frans Kaashoek , Robert Morris, Resilient overlay networks, Proceedings of the eighteenth ACM symposium on Operating systems principles, October 21-24, 2001, Banff, Alberta, Canada
• 2: Suman Banerjee , Bobby Bhattacharjee , Christopher Kommareddy, Scalable application layer multicast, Proceedings of the 2002 conference on Applications, technologies, architectures, and protocols for computer communications, August 19-23, 2002, Pittsburgh, Pennsylvania, USA
• 3: S. Banerjee, T. G. Griffin, and M. Pias. The Interdomain Connectivity of PlanetLab Nodes. In Proc. of PAM, 2004.
• 4: Andy Bavier , Nick Feamster , Mark Huang , Larry Peterson , Jennifer Rexford, In VINI veritas: realistic and controlled network experimentation, Proceedings of the 2006 conference on Applications, technologies, architectures, and protocols for computer communications, September 11-15, 2006, Pisa, Italy
• 5: Coen Bron , Joep Kerbosch, Algorithm 457: finding all cliques of an undirected graph, Communications of the ACM, v.16 n.9, p.575-577, Sept. 1973 [doi>10.1145/362342.362367]
• 6: C. Dovrolis, P. Ramanathan, and D. Moore. What Do Packet Dispersion Techniques Measure? In Proc. of IEEE Infocom, 2001.
• 7: K. P. Gummadi, H. Madhyastha, S. D. Gribble, H. M. Levy, and D. J. Wetherall. Improving the Reliability of Internet Paths with One-hop Source Routing. In Proc. of OSDI, 2004.
• 8: Ningning Hu , Li (Erran) Li , Zhuoqing Morley Mao , Peter Steenkiste , Jia Wang, Locating internet bottlenecks: algorithms, measurements, and implications, Proceedings of the 2004 conference on Applications, technologies, architectures, and protocols for computer communications, August 30-September 03, 2004, Portland, Oregon, USA
• 9: Yang-hua Chu , Sanjay G. Rao , Hui Zhang, A case for end system multicast (keynote address), Proceedings of the 2000 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, p.1-12, June 18-21, 2000, Santa Clara, California, United States
• 10: Ratul Mahajan , Neil Spring , David Wetherall , Thomas Anderson, User-level internet path diagnosis, Proceedings of the nineteenth ACM symposium on Operating systems principles, October 19-22, 2003, Bolton Landing, NY, USA
• 11: T. S. E. Ng and H. Zhang. Predicting Internet Network Distance with Coordinates-Based Approaches. In Proceedings of IEEE INFOCOM, June 2002.
• 12: U. of Oregon Route Views Archive Project. http://www.routeviews.org.
• 13: K. Park, V. S. Pai, L. Peterson, and Z. Wang. CoDNS: Improving DNS Performance and Reliability via Cooperative Lookups. In Proc. of OSDI, 2004.
• 14: Vern Paxson, Strategies for sound internet measurement, Proceedings of the 4th ACM SIGCOMM conference on Internet measurement, October 25-27, 2004, Taormina, Sicily, Italy [doi>10.1145/1028788.1028824]
• 15: L. Peterson, T. Anderson, D. Culler, and T. Roscoe. A Blueprint for Introducing Disruptive Technology Into the Internet. In Proc. of ACM HotNets, 2002.
• 16: PlanetLab. http://www.planet-lab.org.
• 17: Sean Rhea , Brighten Godfrey , Brad Karp , John Kubiatowicz , Sylvia Ratnasamy , Scott Shenker , Ion Stoica , Harlan Yu, OpenDHT: a public DHT service and its uses, Proceedings of the 2005 conference on Applications, technologies, architectures, and protocols for computer communications, August 22-26, 2005, Philadelphia, Pennsylvania, USA
• 18: RipeNCC: Routing Information Service Raw Data. http://abcoude.ripe.net/ris/rawdata/.
• 19: RON. http://nms.csail.mit.edu/ron/sites/.
• 20: Antony I. T. Rowstron , Peter Druschel, Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems, Proceedings of the IFIP/ACM International Conference on Distributed Systems Platforms Heidelberg, p.329-350, November 12-16, 2001
• 21: Antony Rowstron , Peter Druschel, Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility, Proceedings of the eighteenth ACM symposium on Operating systems principles, October 21-24, 2001, Banff, Alberta, Canada
• 22: Joel Sommers , Paul Barford , Nick Duffield , Amos Ron, Improving accuracy in end-to-end packet loss measurement, Proceedings of the 2005 conference on Applications, technologies, architectures, and protocols for computer communications, August 22-26, 2005, Philadelphia, Pennsylvania, USA
• 23: N. Spring, R. Mahajan, and T. Anderson. Quantifying the causes of internet path inflation. In Proc. of SIGCOMM, 2003.
• 24: Jacob Strauss , Dina Katabi , Frans Kaashoek, A measurement study of available bandwidth estimation tools, Proceedings of the 3rd ACM SIGCOMM conference on Internet measurement, October 27-29, 2003, Miami Beach, FL, USA [doi>10.1145/948205.948211]
• 25: L. Subramanian, I. Stoica, H. Balakrishnan, and R. Katz. OverQoS: An Overlay Based Architecture for Enhancing Internet QoS. In Proc. of USENIX NSDI, 2004.
• 26: L. Wang, K. Park, R. Pang, V. S. Pai, and L. Peterson. Reliability and security in the codeen content distribution network. In Proc. of USENIX ATC, 2004.
• 27: Brian White , Jay Lepreau , Leigh Stoller , Robert Ricci , Shashi Guruprasad , Mac Newbold , Mike Hibler , Chad Barb , Abhijeet Joglekar, An integrated experimental environment for distributed systems and networks, Proceedings of the 5th symposium on Operating systems design and implementation

  Due to copyright restrictions we are not able to make the PDFs for this conference available for downloading

  , December 09-11, 2002, Boston, Massachusetts [doi>10.1145/1060289.1060313]
• 28: Praveen Yalagandula , Puneet Sharma , Sujata Banerjee , Sujoy Basu , Sung-Ju Lee, S³: a scalable sensing service for monitoring large networked systems, Proceedings of the 2006 SIGCOMM workshop on Internet network management, p.71-76, September 11-15, 2006, Pisa, Italy [doi>10.1145/1162638.1162650]
• 29: R. Zhang and Y. C. Hu. Assisted Peer-to-Peer Search with Partial Indexing. In Proc. of IEEE INFOCOM, 2005.
• 30: GNP Homepage. http://www.cs.rice.edu/ Eugeneng/research/gnp/.
• 31: Iperf. http://dast.nlanr.net/Projects/Iperf/.
• 32: PlanetLab IPerf. http://www.planet-lab.org/logs/iperf/.
• 33: S3: Scalable Sensing Service. http://networking.hpl.hp.com/s-cube/.
• 34: The P2PSim Project. http://pdos.csail.mit.edu/p2psim/.
• 35: The Gnutella protocol specification, 2000. http://dss.clip2.com/GnutellaProtocol04.pdf.

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.1 Network Architecture and Design

General Terms:
Design, Experimentation, Measurement, Performance

Keywords:
network characteristics, testbeds