Impact of simulation tool on TCP performance results: a case study with ns-2 and Opnet

ABSTRACT

Network researchers and designers rely on network simulation tools to do performance evaluation. There are a number of different network simulation tools, each with a set of publicly available models. Two of the more popular tools, ns-2 and Opnet have a large library of available models. The existing models can be used to configure simulation experiments in a relatively quick and easy manner. The choice of simulation tool often depends on the availability of models as well as the user's preference. Most users assume that performance results on different simulators will be comparable, particularly when using models that have been widely used and tested. Using simple simulation experiments, this paper shows that TCP performance results obtained from ns-2 and Opnet can have significant differences. The differences are exposed when TCP is tested with error rates ranging from 1% packet loss to 20% packet loss. A study of the actions taken by TCP as a result of the losses provides insight into the performance differences.

REFERENCES

• 1: David Cavin , Yoav Sasson , André Schiper, On the accuracy of MANET simulators, Proceedings of the second ACM international workshop on Principles of mobile computing, October 30-31, 2002, Toulouse, France [doi>10.1145/584490.584499]
• 2: {Emmelmann 2002} Emmelmann, M., "Effects of Advertised Receive Buffer Size and Timer Granularity on TCP Performance over Erroneous Links in a LEO Satellite Network", IEEE Globecom 22, Vol. 3, Page(s):2955 -- 2958, 2002.
• 3: {Fall 2005} Fall, K., and Varadhan, K., "The ns Manual", 2005.
• 4: {Jain 1991} Jain, R., "The Art of Computer Systems Performance Analysis", John Wiley & Sons Inc., 1991.
• 5: {Jiang 2003} Jiang, R., Gupta, V., Ravishankar, C. V., "Interactions between TCP and the IEEE 802.11 MAC protocol", 2003 Proceedings of DARPA Information Survivability Conference and Exposition, Vol. 1, Page(s):273 -- 282, 2003.
• 6: {Lucio 2003} Lucio, G. F., Parades-Farrera, M., Jammeh, E., Fleury, M., Reed, M. J., "OPNET Modeler and NS-2:Comparing the Accuracy of Network Simulators for Packet-Level Analysis using a Network Testbed", Electronic System Engineering Department, University of Essex, United Kingdom, July 2003.
• 7: {Opnet} http://www.opnet.com
• 8: {Pawlikowski 2002} Pawlikowski, K., Jeong, H-d., and Lee, J.-S., "On credibility of simulation studies of telecommunication networks", IEEE Communication Magazine, Vol. 40, Issue 1, Page(s):132--139, 2002.
• 9: Shugong Xu , T. Saadawi , Myung Lee, Comparison of TCP Reno and Vegas in wireless mobile ad hoc networks, Proceedings of the 25th Annual IEEE Conference on Local Computer Networks, p.42, November 08-10, 2000
• 10: Xiang Zeng , Rajive Bagrodia , Mario Gerla, GloMoSim: a library for parallel simulation of large-scale wireless networks, Proceedings of the twelfth workshop on Parallel and distributed simulation, p.154-161, May 26-29, 1998, Banff, Alberta, Canada

INDEX TERMS

Keywords:
Opnet, TCP performance, error rate, model implementation, ns-2