Automating software module testing for FAA certification

ABSTRACT

Structural testing, also called code-based or white-box testing, is performed for the purpose of exercising the code thoroughly. The code coverage measurements are used to evaluate the effectiveness of the test cases toward this goal. Manual, code-based testing is cumbersome and time-consuming. Automation provides help in selecting test data, applying those test cases to the software, and deciding whether a program has been tested enough by providing coverage metrics. This paper describes an automated module testing process for software written in Ada using a tool set called Test Set Editor (TSE). TSE consists of a proprietary set of programs which work in combination with Microsoft Excel spreadsheet program and home grown test scripts written in Tcl/tk. Use of the tool set is shown to yield significant cost savings in constructing structural tests needed to satisfy FAA certification requirements for safety-critical software.

REFERENCES

• 1: Tsuneo Yamaura, How to Design Practical Test Cases, IEEE Software, v.15 n.6, p.30-36, November 1998 [doi>10.1109/52.730835]
• 2: Barry Boehm , Victor R. Basili, Software Defect Reduction Top 10 List, Computer, v.34 n.1, p.135-137, January 2001 [doi>10.1109/2.962984]
• 3: James A. Whittaker, What Is Software Testing? And Why Is It So Hard?, IEEE Software, v.17 n.1, p.70-79, January 2000 [doi>10.1109/52.819971]
• 4: Thomas Ball , James R. Larus, Using Paths to Measure, Explain, and Enhance Program Behavior, Computer, v.33 n.7, p.57-65, July 2000 [doi>10.1109/2.869371]
• 5: Jairus Hihn , Hamid Habib-agahi, Cost estimation of software intensive projects: a survey of current practices, Proceedings of the 13th international conference on Software engineering, p.276-287, May 13-17, 1991, Austin, Texas, United States
• 6: RTCA/DO-178B: Software Considerations in Airborne Systems and Equipment Certification, RTCA, Inc., Washington, D.C., December 1992.
• 7: Chilenski, J. J., Miller, S. P. Applicability of Modified Condition Decision Coverage to Software Testing. Software Engineering Journal, v.7, n.5, September 1994, 193-200.

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging
          Subjects: Code inspections and walk-throughs

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.2 Design Tools and Techniques
          Subjects: Modules and interfaces

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.2 Automatic Programming
          Subjects: Program verification

General Terms:
Verification

Keywords:
FAA Certification, Software Module Test, Structural Test, Test Automation