Blowtorch: a framework for firewall test automation

ABSTRACT

Firewalls play a crucial role in network security. Experience has shown that the development of firewall rule sets is complex and error prone. Rule set errors can be costly, by allowing damaging traffic in or by blocking legitimate traffic and causing essential applications to fail. Consequently, firewall testing is extremely important. Unfortunately, it is also hard and there is little tool support available.Blowtorch is a C++ framework for firewall test generation. The central construct is the packet iterator: an event-driven generator of timestamped packet streams. Blowtorch supports the development of packet iterators with a library for packet header creation and parsing, a transmit scheduler for multiplexing of multiple packet streams, and a receive monitor for demultiplexing of arriving packet streams. The framework provides iterators which generate packet streams using covering arrays, production grammars, and replay of captured TCP traffic. Blowtorch has been used to develop tests for industrial firewalls that are placed between an IT network and a process control network.

REFERENCES

• 1: Protos - security testing of protocol implementations, 2000. http://www.ee.oulu./research/ouspg/protos/.
• 2: Khalid Al-Tawil , Ibrahim A. Al-Kaltham, Evaluation and testing of internet firewalls, International Journal of Network Management, v.9 n.3, p.135-149, May–June 1999 [doi>10.1002/(SICI)1099-1190(199905/06)9:3<135::AID-NEM311>3.0.CO;2-5]
• 3: E. Byres and K. Savage. NISCC good practice guide on rewall deployment for SCADA and process control networks. http://www.niscc.gov.uk/niscc/docs/re20050223-00157.pdf, 2005.
• 4: David M. Cohen , Siddhartha R. Dalal , Jesse Parelius , Gardner C. Patton, The Combinatorial Design Approach to Automatic Test Generation, IEEE Software, v.13 n.5, p.83-88, September 1996 [doi>10.1109/52.536462]
• 5: A. G. Duncan , J. S. Hutchison, Using attributed grammars to test designs and implementations, Proceedings of the 5th international conference on Software engineering, p.170-178, March 09-12, 1981, San Diego, California, United States
• 6: D. Hoffman and E. Byres. Worlds in collision: Ethernet on the plant oor. In ISA Emerging Technologies Conference. Instrumentation Systems and Automation Society, Oct. 2002.
• 7: Jan Jürjens , Guido Wimmel, Specification-Based Testing of Firewalls, Revised Papers from the 4th International Andrei Ershov Memorial Conference on Perspectives of System Informatics: Akademgorodok, Novosibirsk, Russia, p.308-316, July 02-06, 2001
• 8: MicroC/OS-II: the real-time kernel, CMP Media, Inc., 2002
• 9: Alain Mayer , Avishai Wool , Elisha Ziskind, Fang: A Firewall Analysis Engine, Proceedings of the 2000 IEEE Symposium on Security and Privacy, p.177, May 14-17, 2000
• 10: Bill McCarty , Mark J. Cox, Red Hat Linux Firewalls, John Wiley & Sons, Inc., New York, NY, 2002
• 11: Joel Sommers , Vinod Yegneswaran , Paul Barford, A framework for malicious workload generation, Proceedings of the 4th ACM SIGCOMM conference on Internet measurement, October 25-27, 2004, Taormina, Sicily, Italy [doi>10.1145/1028788.1028799]
• 12: K. C. Tai , Y. Lie, A Test Generation Strategy for Pairwise Testing, IEEE Transactions on Software Engineering, v.28 n.1, p.109-111, January 2002 [doi>10.1109/32.979992]
• 13: Andrew Tanenbaum, Computer Networks, Prentice Hall Professional Technical Reference, 2002
• 14: A Quantitative Study of Firewall Configuration Errors, Computer, v.37 n.6, p.62-67, June 2004 [doi>10.1109/MC.2004.2]

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging
          Subjects: Testing tools (e.g., data generators, coverage testing)

General Terms:
Security, Verification

Keywords:
automated testing, capture/replay, covering array, network firewall, production grammar