Automatic testing of TCP/IP implementations using QuickCheck

ABSTRACT

We describe how to use model based testing for testing a network stack. We present a framework that together with the property based testing tool QuickCheck can be used to test the TCP layer of the Internet protocol stack. TCP is a rather difficult protocol to test, since it hides a lot of operations for the user that communicates to the stack via a socket interface. Internally, a lot happens and by only controlling the interface, full testing is not possible. This is typical for more complex protocols and we therefore claim that the presented method can easily be extended to other cases. We present an automatic test case generator for TCP using Quickcheck. This tester generates packet flows to test specific features of a TCP stack. It then controls the stack under test to run the test by using the interface provided by it (for example, the socket interface), and by sending replies to the packets created by the stack under test. We validated the test framework on the standard Linux TCP/IP implementation.

REFERENCES

• 1: Thomas Arts, John Hughes, Joakim Johansson, and Ulf Wiger. Testing Telecoms Software with Quviq QuickCheck. In ERLANG'06: Proceedings of the 2006 ACM SIGPLAN workshop on Erlang, New York, NY, USA, 2006. ACM Press.
• 2: Koen Claessen and John Hughes. QuickCheck: a lightweight tool for random testing of Haskell programs. In ICFP '00: Proceedings of the fifth ACM SIGPLAN international conference on Functional programming, pages 268--279, New York, NY, USA, 2000. ACM.
• 3: John Hughes. Quickcheck Testing for Fun and Profit. In Michael Hanus, editor, 9th International Symposium on Practical Aspects of Declarative Languages. Springer, 2007.
• 4: Information Sciences Institute. RFC 793: Transmission control protocol, 1981. URL http://rfc.sunsite.dk/rfc/rfc793.html. Edited by Jon Postel. Available at http://rfc.sunsite.dk/rfc/rfc793.html.
• 5: Van Jacobson, Craig Leres, and Steven McCanne. libpcap: Packet capture library. URL http://www.tcpdump.org.
• 6: Javier Paris, Victor Gulias, and Alberto Valderruten. A high performance erlang TCP/IP stack. In ERLANG '05: Proceedings of the 2005 ACM SIGPLAN workshop on Erlang, pages 52--61, New York, NY, USA, 2005. ACM. ISBN 1-59593-066-3. doi: http://doi.acm.org/10.1145/1088361.1088372.
• 7: Raid Zaghal and Javed Khan. EFSM/SDL modeling of the original TCP standard (RFC793) and the Congestion Control Mechanism of TCP Reno. URL: http://www.medianet.kent.edu/technicalreports.html.

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging
          Subjects: Testing tools (e.g., data generators, coverage testing)

General Terms:
Reliability

Keywords:
networking, tcp/ip, testing