Safety and Software Intensive Systems: Challenges Old and New

ABSTRACT

There is an increased use of software in safety-critical systems; a trend that is likely to continue in the future. Although traditional system safety techniques are applicable to software intensive systems, there are new challenges emerging. In this report we will address four issues we believe will pose challenges in the future. First, the nature of safety is continuing to be widely misunderstood and known system safety techniques are not applied. Second, our ability to demonstrate (certify) that safety requirements have been met is inadequate. Third, modeling and automated tools, for example, code generation and automated testing, are introduced in a hope to increase productivity; this reliance on tools rather than people, however, introduces new and poorly understood problems. Finally, safety-critical systems are increasingly relying on data (configuration data or databases), incorrect data could have catastrophic and widespread consequences.

REFERENCES

• 1: Ilan Beer , Shoham Ben-David , Cindy Eisner , Yoav Rodeh, Efficient Detection of Vacuity in Temporal Model Checking, Formal Methods in System Design, v.18 n.2, p.141-163, March 1, 2001 [doi>10.1023/A:1008779610539]
• 2: Antonia Bertolino, Software Testing Research: Achievements, Challenges, Dreams, 2007 Future of Software Engineering, p.85-103, May 23-25, 2007 [doi>10.1109/FOSE.2007.25]
• 3: [3] P. Bishop and R. Bloomfield. A methodology for safety case development. In F. Redmill and T. Anderson, editors, Industrial Perspectives of Safety-critical Systems: Proceedings of the Sixth Safety-critical Systems Symposium, pages 194-203. Springer, 1998.
• 4: [4] M. R. Blackburn, R. D. Busser, and J. S. Fontaine. Automatic generation of test vectors for SCR-style specifications. In Proceedings of the 12th Annual Conference on Computer Assurance, COMPASS'97, June 1997.
• 5: [5] The Future of Drug Safety: Promoting and Protecting the Health of the Public. Board on Population Health and Public Health Practice (BPH), Institute of Medicine (IOM), 2006.
• 6: [6] L. Briand and A. Wolf, editors. Future of Software Engineering 2007. IEEE-CS Press, 2007.
• 7: S. S. Brilliant , J. C. Knight , N. G. Leveson, The Consistent Comparison Problem in N-Version Software, IEEE Transactions on Software Engineering, v.15 n.11, p.1481-1485, November 1989 [doi>10.1109/32.41339]
• 8: Susan S. Brilliant , John C. Knight , Nancy G. Leveson, Analysis of Faults in an N-Version Software Experiment, IEEE Transactions on Software Engineering, v.16 n.2, p.238-247, February 1990 [doi>10.1109/32.44387]
• 9: Frederick P. Brooks, Jr., No Silver Bullet Essence and Accidents of Software Engineering, Computer, v.20 n.4, p.10-19, April 1987 [doi>10.1109/MC.1987.1663532]
• 10: Frederick P. Brooks, Jr., The mythical man-month (anniversary ed.), Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1995
• 11: [11] M. Chechik, M. Gheorghiu, and A. Gurfinkel. Finding environmental guarantees. In Proceedings of Fundamental Approaches to Software Engineering (FASE'07), To appear in 2007.
• 12: [12] L. Chen and A. Avizienis. N-version programming: A fault-tolerance approach to reliability of software operation. In Digest of Papers FTCS-8: Eighth Annual International Conference on Fault Tolerant Computing, pages pp. 3-9, Tolouse, France, June 1978.
• 13: Betty H. C. Cheng , Joanne M. Atlee, Research Directions in Requirements Engineering, 2007 Future of Software Engineering, p.285-303, May 23-25, 2007 [doi>10.1109/FOSE.2007.17]
• 14: [14] J. Chilenski and S. Miller. Applicability of modified condition/decision coverage to software testing. Software Engineering Journal, 9:193-200, September 1994.
• 15: Hana Chockler , Orna Kupferman , Robert P. Kurshan , Moshe Y. Vardi, A Practical Approach to Coverage in Model Checking, Proceedings of the 13th International Conference on Computer Aided Verification, p.66-78, July 18-22, 2001
• 16: Hana Chockler , Orna Kupferman , Moshe Y. Vardi, Coverage Metrics for Temporal Logic Model Checking, Proceedings of the 7th International Conference on Tools and Algorithms for the Construction and Analysis of Systems, p.528-542, April 02-06, 2001
• 17: Edmund M. Clarke, Jr. , Orna Grumberg , Doron A. Peled, Model checking, MIT Press, Cambridge, MA, 2000
• 18: Matthew B. Dwyer , John Hatcliff , Robby Robby , Corina S. Pasareanu , Willem Visser, Formal Software Analysis Emerging Trends in Software Model Checking, 2007 Future of Software Engineering, p.120-136, May 23-25, 2007 [doi>10.1109/FOSE.2007.6]
• 19: [19] Esterel-Technologies. Corporate web page. www.esterel-technologies.com, 2004.
• 20: [20] Esterel-Technologies. SCADE Suite product description. http://www.esterel-technologies.com/v2/ scadeSuite-ForSafetyCriticalSoftwareDevelopment/index.html, 2004.
• 21: [21] Software Tools Workshop of FAA and Embry-Riddle Aeronautical University. http://www.erau.edu/db/campus/softwaretoolsforum.html, May 2004.
• 22: Proceedings of the Conference on The Future of Software Engineering, May 2000
• 23: Robert France , Bernhard Rumpe, Model-driven Development of Complex Software: A Research Roadmap, 2007 Future of Software Engineering, p.37-54, May 23-25, 2007 [doi>10.1109/FOSE.2007.14]
• 24: [24] W. S. Greenwell, J. C. Knight, C. M. Holloway, and J. J. Pease. A taxonomy of fallacies in system safety arguments. In Proceedings of the 2006 International System Safety Conference , 2006.
• 25: David Harel , Amir Pnueli , Hagi Lachover , Amnon Naamad , Michal Politi , Rivi Sherman , Aharon Shtull-Trauring , Mark Trakhtenbrot, STATEMATE: A Working Environment for the Development of Complex Reactive Systems, IEEE Transactions on Software Engineering, v.16 n.4, p.403-414, April 1990 [doi>10.1109/32.54292]
• 26: Mats P. E. Heimdahl , Devaraj George, Test-Suite Reduction for Model Based Tests: Effects on Test Quality and Implications for Testing, Proceedings of the 19th IEEE international conference on Automated software engineering, p.176-185, September 20-24, 2004 [doi>10.1109/ASE.2004.67]
• 27: [27] M. P. Heimdahl, G. Devaraj, and R. J. Weber. Specification test coverage adequacy criteria = specification test generation inadequacy criteria? In Proceedings of the Eighth IEEE International Symposium on High Assurance Systems Engineering (HASE), Tampa, Florida, March 2004.
• 28: [28] M. P. Heimdahl, S. Rayadurgam, W. Visser, G. Devaraj, and J. Gao. Auto-generating test sequences using model checkers: A case study. In 3rd International Worshop on Formal Approaches to Testing of Software (FATES 2003), 2003.
• 29: [29] K. Heninger. Specifying software requirements for complex systems: New techniques and their application. IEEE Transactions on Software Engineering, 6(1):2-13, Januaray 1980.
• 30: Debra S. Herrmann, Software safety and reliability: techniques, approaches, and standards of key industrial sectors, IEEE Computer Society, Washington, DC, 1999
• 31: Gerard J. Holzmann, The Model Checker SPIN, IEEE Transactions on Software Engineering, v.23 n.5, p.279-295, May 1997 [doi>10.1109/32.588521]
• 32: Hyoung Seok Hong , Sung Deok Cha , Insup Lee , Oleg Sokolsky , Hasan Ural, Data flow testing as model checking, Proceedings of the 25th International Conference on Software Engineering, May 03-10, 2003, Portland, Oregon
• 33: Hyoung Seok Hong , Insup Lee , Oleg Sokolsky , Hasan Ural, A Temporal Logic Based Theory of Test Coverage and Generation, Proceedings of the 8th International Conference on Tools and Algorithms for the Construction and Analysis of Systems, p.327-341, April 08-12, 2002
• 34: [34] IEC-61508: Functional Safety of Electrical/ Electronic/ Programmable Electronic Safety-Related Systems. International Electrotechnical Commission (IEC), 1999.
• 35: Matthew S. Jaffe , Nancy G. Leveson , Mats P. E. Heimdahl , Bonnie E. Melhart, Software Requirements Analysis for Real-Time Process-Control Systems, IEEE Transactions on Software Engineering, v.17 n.3, p.241-258, March 1991 [doi>10.1109/32.75414]
• 36: [36] T. P. Kelly and R. A. Weaver. The goal structuring notation-a safety argument notation. In Proceedings of the Dependable Systems and Networks 2004 Workshop on Assurance Cases, 2004.
• 37: J. C. Knight , N. G. Leveson, An experimental evaluation of the assumption of independence in multiversion programming, IEEE Transactions on Software Engineering, v.12 n.1, p.96-109, Jan. 1986
• 38: John C. Knight, Focusing software education on engineering, ACM SIGSOFT Software Engineering Notes, v.30 n.2, March 2005 [doi>10.1145/1050849.1050852]
• 39: John C. Knight , Nancy G. Leveson, Should software engineers be licensed?, Communications of the ACM, v.45 n.11, November 2002 [doi>10.1145/581571.581601]
• 40: John C. Knight , Nancy G. Leveson, Software and higher education, Communications of the ACM, v.49 n.1, January 2006 [doi>10.1145/1107458.1107486]
• 41: [41] J. C. Knight, E. A. Strunk, W. S. Greenwell, and K. S. Wasson. Specification and analysis of data for safety-critical systems. In 22nd International System Safety Conference, Providence, RI, August 2004.
• 42: [42] O. Kupferman and M. Y. Vardi. Vacuity detection in temporal model checking. Journal on Software Tools for Technology Transfer, 4(2), February 2003.
• 43: Timothy C. Lethbridge , Jorge Diaz-Herrera , Richard J. Jr. LeBlanc , J. Barrie Thompson, Improving software practice through education: Challenges and future trends, 2007 Future of Software Engineering, p.12-28, May 23-25, 2007 [doi>10.1109/FOSE.2007.13]
• 44: Nancy G. Leveson , Mats Per Erik Heimdahl , Holly Hildreth , Jon D. Reese, Requirements Specification for Process-Control Systems, IEEE Transactions on Software Engineering, v.20 n.9, p.684-707, September 1994 [doi>10.1109/32.317428]
• 45: [45] N. Leveson, J. Reese, and M. Heimdahl. SpecTRM: A CAD system for digital automation. In Proceedings of the 17th Digital Avionics Systems Conference, November 1998.
• 46: Nancy G. Leveson, Software safety: why, what, and how, ACM Computing Surveys (CSUR), v.18 n.2, p.125-163, June 1986 [doi>10.1145/7474.7528]
• 47: Nancy G. Leveson, Safeware: system safety and computers, ACM, New York, NY, 1995
• 48: [48] N. G. Leveson. System Safety Engineering: Back To The Future. On line publication: http://sunnyday.mit.edu/book2.pdf, 2002.
• 49: [49] N. G. Leveson. A new approach to hazard analysis for complex systems. In Proceedings of the International Conference of the System Safety Society, Ottawa, Canada, August 2003.
• 50: Nancy G. Leveson, A Systems-Theoretic Approach to Safety in Software-Intensive Systems, IEEE Transactions on Dependable and Secure Computing, v.1 n.1, p.66-86, January 2004 [doi>10.1109/TDSC.2004.1]
• 51: Nancy G. Leveson , Mats P. E. Heimdahl , Jon Damon Reese, Designing specification languages for process control systems: lessons learned and steps to the future, Proceedings of the 7th European software engineering conference held jointly with the 7th ACM SIGSOFT international symposium on Foundations of software engineering, p.127-145, September 06-10, 1999, Toulouse, France
• 52: Robyn R. Lutz, Targeting safety-related errors during software requirements analysis, Journal of Systems and Software, v.34 n.3, p.223-230, Sept. 1996 [doi>10.1016/0164-1212(95)00077-1]
• 53: Robyn R. Lutz, Software engineering for safety: a roadmap, Proceedings of the Conference on The Future of Software Engineering, p.213-226, June 04-11, 2000, Limerick, Ireland [doi>10.1145/336512.336556]
• 54: [54] MathWorks. The MathWorks Inc. corporate web page. http://www.mathworks.com, 2004.
• 55: [55] Mathworks Inc. Simulink product web site. Via the world-wide-web: http://www.mathworks.com.
• 56: [56] Mathworks Inc. Stateflow product web site. vVia the world-wide-web: http://www.mathworks.com.
• 57: John A McDermid, Software safety: where's the evidence?, Proceedings of the Sixth Australian workshop on Safety critical systems and software, p.1-6, July 01, 2001, Brisbane, Australia
• 58: [58] J. A. McDermid and D. J. Pumfrey. Software safety: Why is there no consensus? In Proceedings of the 19th International System Safety Conference. System Safety Society, 2001.
• 59: [59] Merck. Vioxx home page. http://www.vioxx.com/, 2004.
• 60: [60] S. Miller, A. Tribble, T. Carlson, and E. J. Danielson. Flight guidance system requirements specification. Technical Report CR-2003-212426, NASA, June 2003.
• 61: Steven P. Miller , Alan C. Tribble , Michael W. Whalen , Mats P. E. Heimdahl, Proving the shalls: Early validation of requirements through formal methods, International Journal on Software Tools for Technology Transfer (STTT), v.8 n.4, p.303-319, August 2006
• 62: [62] Requirements for Safety Related Software in Defence Equipment, Issue 2. UK Ministry of Defence, 1997.
• 63: [63] Safety Management Requirements for Defence Systems, Issue 2. UK Ministry of Defence, 1996.
• 64: [64] The NuSMV Toolset, 2005. Available at http://nusmv.irst.itc.it/.
• 65: [65] S. Owre, N. Shankar, and J. Rushby. User Guide for the PVS Specification and Verification System. Computer Scienc eLaboratory; SRI International, Menlo Park, CA 94025, beta release edition, March 1993.
• 66: David Lorge Parnas, Education for Computing Professionals, Computer, v.23 n.1, p.17-22, January 1990 [doi>10.1109/2.48796]
• 67: Mitra Purandare , Fabio Somenzi, Vacuum Cleaning CTL Formulae, Proceedings of the 14th International Conference on Computer Aided Verification, p.485-499, July 27-31, 2002
• 68: [68] D. Raheja. Assurance Technologies: Principles and Practices . McGraw-Hill, 1991.
• 69: [69] S. Rayadurgam and M. P. Heimdahl. Coverage based test-case generation using model checkers. In Proceedings of the 8th Annual IEEE International Conference and Workshop on the Engineering of Computer Based Systems (ECBS 2001), pages 83-91. IEEE Computer Society, April 2001.
• 70: [70] R. I. C. RI and M. F. O'Connor. Medication Safety: A Guide to Health Care Facilities, chapter Thinking about accidents and systems, pages 73-87. American Society of Health-System Pharmacists, Bethesda, MD, 2005.
• 71: [71] RTCA. DO-178B: Software Considerations In Airborne Systems and Equipment Certification. RTCA, 1992.
• 72: James Rumbaugh , Ivar Jacobson , Grady Booch, The Unified Modeling Language reference manual, Addison-Wesley Longman Ltd., Essex, UK, 1998
• 73: [73] SAE-ARP4761. Guidelines and Methods for Conducting the Safety Assessment Process on Civil Airborne Systems and Equipment. SAE International, December 1996.
• 74: [74] RTCA SC-205 (Joint with EUROCAE WG-71) Software Considerations. http://www.rtca.org/comm/Committee.cfm?id=55.
• 75: M. L. Shooman, Avionics software problem occurrence rates, Proceedings of the The Seventh International Symposium on Software Reliability Engineering (ISSRE '96), p.55, October 30-November 02, 1996
• 76: Neil R. Storey, Safety Critical Computer Systems, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1996
• 77: [77] N. Storey and A. Faulkner. The characteristics of data in data-intensive safety-related systems. In SAFECOMP, pages 396-409, 2003.
• 78: [78] N. Storey and A. Faulkner. Data--the forgotten system component? Journal of System Safety, Vol. 39(No. 4):10-14, 36, 2003.
• 79: Richard N. Taylor , Andre van der Hoek, Software Design and Architecture The once and future focus of software engineering, 2007 Future of Software Engineering, p.226-243, May 23-25, 2007 [doi>10.1109/FOSE.2007.21]
• 80: Michael W. Whalen , Ajitha Rajan , Mats P.E. Heimdahl , Steven P. Miller, Coverage metrics for requirements-based testing, Proceedings of the 2006 international symposium on Software testing and analysis, July 17-20, 2006, Portland, Maine, USA [doi>10.1145/1146238.1146242]
• 81: [81] E. V. Wyk and M. P. Heimdahl. Flexibility in modeling languages and tools: A call to arms. In Proceedings of the IEEE ISoLA Workshop on Leveraging Applications of Formal Methods, Verification, and Validation, Columbia, Maryland, USA, September 2005.
• 82: Marc K. Zimmerman , Kristina Lundqvist , Nancy Leveson, Investigating the readability of state-based formal requirements specification languages, Proceedings of the 24th International Conference on Software Engineering, May 19-25, 2002, Orlando, Florida [doi>10.1145/581339.581347]

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.0 General
          Subjects: Protection mechanisms

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging
  D.3 PROGRAMMING LANGUAGES
      D.3.4 Processors
          Subjects: Code generation

General Terms:
Design, Security, Verification