A highly configurable test system for evolutionary black-box testing of embedded systems

ABSTRACT

During the development of electronic control units (ECU) in domains like the automotive industry, tests are performed on various test platforms, such as model-in-the-loop, software-in-the-loop, processor-in-the-loop, and hardware-in-the-loop platforms in order to find faults in early development stages. Test cases must be specified to verify the properties demanded of the system on these test platforms. This is an expensive and non-trivial task. Evolutionary black-box testing, a recent approach to automating the creation of interesting test cases, can solve this task completely automatically.

This paper describes our evolutionary test system and how to apply it to the test of functional and non-functional properties of embedded systems. Our system supports the aforementioned test platforms and allows for the reuse of the generated test cases across them. In a case study with an antilock braking system, we demonstrate the operation of the system.

REFERENCES

• 1: Berner&Mattner Systemtechnik GmbH MESSINA. http://berner-mattner.com/automotive-messina.php
• 2: Berner&Mattner Systemtechnik GmbH modularHiL http://berner-mattner.com/automotive-modulhil.php
• 3: Da Costa, L.; Schoenauer, M.: GUIDE, a Graphical User Interface for Evolutionary Algorithms. GECCO Workshop on Open-Source Software for Applied Genetic and Evolutionary Computation (SoftGEC), 2007.
• 4: Dimitar, M.; Dimitrov, I. M.; Spasov, I.: Evotest -- Framework for customizable implementation of Evolutionary Testing. International Workshop on Software and Services, October 2008, Sofia, Bulgaria.
• 5: EvoTest http://www.evotest.eu/
• 6: GUIDE. http://guide.gforge.inria.fr/
• 7: Jones, B.F., Eyres, D.E. and Sthamer, H.-H.: A Strategy for using Genetic Algorithms to Automate Branch and Fault-based Testing. The Computer Journal, vol. 41, no. 2, pp. 98--107, 1998.
• 8: PikeTec GmbH TPT. http://piketec.com/products/tpt.php?lang=en
• 9: Sthamer, H.-H.: The Automatic Generation of Software Test Data Using Genetic Algorithms. PhD Thesis, University of Glamorgan, Pontyprid, Wales, Great Britain, 1996.
• 10: Tesis veDYNA. http://www.tesis.de/en/index.php?page=544
• 11: Joachim Wegener , Matthias Grochtmann, Verifying Timing Constraints of Real-Time Systems by Meansof Evolutionary Testing, Real-Time Systems, v.15 n.3, p.275-298, Nov. 1998 [doi>10.1023/A:1008096431840]
• 12: Wegener, J., Baresel, A., Sthamer H.: Evolutionary Test Environment for Automatic Structural Testing. Information and Software Technology, Special Issue devoted to the Application of Metaheuristic Algorithms to Problems in Software Engineering, vol. 43, pp. 841--854, 2001.
• 13: Wegener, J.: Evolutionärer Test des Zeitverhaltens von Realzeit-Sytemen (Evolutionary Testing of the Temporal Behaviour of Real-Time Systems). Shaker Verlag, 2001.
• 14: Windisch, A.: Search-Based Testing of Complex Simulink Models containing Stateflow Diagrams. Proceedings of the 1st International Workshop on Search-Based Software Testing, Lillehammer, Norway, 2008.

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging
          Subjects: Testing tools (e.g., data generators, coverage testing)

General Terms:
Design, Documentation, Experimentation, Measurement, Performance, Verification

Keywords:
antilock-braking-system, evolutionary testing, functional testing, hardware-in-the-loop-testing, testing infrastructure