A new approach for software testability analysis

ABSTRACT

Software testability analysis has been an important research direction since 1990s and becomes more pervasive when entering 21st century. In this paper, we summarize problems in existing research work. We propose to use beta distribution to indicate software testability. When incorporating testing effectiveness information, we theoretically prove that the distribution can express testing effort and test value at the same time. We conduct experiment and validate our results on Siemens programs. Future work concentrate on deducing a prior estimation of the distribution for given software and testing criterion pair from program slicing and semantic analysis.

REFERENCES

• 1: N. P. Edwards, The effect of certain modular design principles on testability, ACM SIGPLAN Notices, v.10 n.6, p.401-410, June 1975
• 2: McCall J.A., Richards P.K., and Walters G.F. Factors in Software Quality. NIST, New York, 1977.
• 3: Boehm B.W., C B., and H.E. Characteristics of software quality. TRW, Amsterdam, North Holland, 1978.
• 4: ISO/IEC. ISO/IEC 9126:Software Engineering Product quality. ISO Press, Geneva, Switzerland, 1991.
• 5: Richard Bache , Monika Mullerburg, Measures of testability as a basis for quality assurance, Software Engineering Journal, v.5 n.2, p.86-92, Mar. 1990
• 6: Roy S. Freedman, Testability of Software Components, IEEE Transactions on Software Engineering, v.17 n.6, p.553-564, June 1991 [doi>10.1109/32.87281]
• 7: J.Voas. Factors that affect program testabilities. In Proc. Of the 9th Pacific Northwest Software Quality Conf., pages 235--247. IEEE, October 1991.
• 8: Jeffrey M. Voas , Keith W. Miller, Semantic metrics for software testability, Journal of Systems and Software, v.20 n.3, p.207-216, March 1993 [doi>10.1016/0164-1212(93)90064-5]
• 9: Benoit Baudry , Yves Le Traon , Gerson Sunyé, Testability Analysis of a UML Class Diagram, Proceedings of the 8th International Symposium on Software Metrics, p.54, June 04-07, 2002
• 10: Jeffrey M. Voas, PIE: A Dynamic Failure-Based Technique, IEEE Transactions on Software Engineering, v.18 n.8, p.717-727, August 1992 [doi>10.1109/32.153381]
• 11: B. W. N. Lo , H. Shi, A Preliminary Testability Model for Object-Oriented Software, Proceedings of the 1998 International Conference on Software Engineering: Education & Practice, p.330, January 26-29, 1998
• 12: Jerry Gao , Ming-Chih Shih, A Component Testability Model for Verification and Measurement, Proceedings of the 29th Annual International Computer Software and Applications Conference (COMPSAC'05) Volume 2, p.211-218, July 26-28, 2005
• 13: SeDo Sohn , PoongHyun Seong, Quantitative evaluation of safety critical software testability based on fault tree analysis and entropy, Journal of Systems and Software, v.73 n.2, p.351-360, October 2004 [doi>10.1016/j.jss.2003.10.028]
• 14: J.E.Payne, R.T.Alexander, and C.D. Hutchinson. Design-for-testability for object oriented software. Object Magazine, 7(5):34--43, May 1997.
• 15: A. Goel, S.C Gupta, and S.K. Wasan. Probe mechanism for object-oriented software testing. In Proc. of the 6th International Conference on Fundamental Approaches to Software Engineering, pages 310--324. IEEE, April 2003.
• 16: Antonia Bertolino , Lorenzo Strigini, On the Use of Testability Measures for Dependability Assessment, IEEE Transactions on Software Engineering, v.22 n.2, p.97-108, February 1996 [doi>10.1109/32.485220]
• 17: Mark C. K. Yang , W. Eric Wong , Alberto Pasquini, Applying Testability to Reliability Estimation, Proceedings of the The Ninth International Symposium on Software Reliability Engineering, p.90, November 04-07, 1998
• 18: IEEE. IEEE Standard Glossary of Software Engineering Terminology. IEEE Computer Society Press, New York, 1990.
• 19: Norman Fenton , Shari Lawrence Pfleeger, Software metrics (2nd ed.): a rigorous and practical approach, PWS Publishing Co., Boston, MA, 1997
• 20: Magiel Bruntink , Arie van Deursen, Predicting Class Testability using Object-Oriented Metrics, Proceedings of the Source Code Analysis and Manipulation, Fourth IEEE International Workshop on (SCAM'04), p.136-145, September 15-16, 2004 [doi>10.1109/SCAM.2004.15]
• 21: Anthony Finkelstein , Jeff Kramer, Software engineering: a roadmap, Proceedings of the Conference on The Future of Software Engineering, p.3-22, June 04-11, 2000, Limerick, Ireland [doi>10.1145/336512.336519]
• 22: L.Zhao, J.M. Wang, and J.G. Sun. Improving the effectiveness of software testability measure. In Proc. of ISSADS 2006 (To be appeared), January 2006.
• 23: Norman E. Fenton , Martin Neil, Software metrics: roadmap, Proceedings of the Conference on The Future of Software Engineering, p.357-370, June 04-11, 2000, Limerick, Ireland [doi>10.1145/336512.336588]
• 24: Bev Littlewood , Lorenzo Strigini, Software reliability and dependability: a roadmap, Proceedings of the Conference on The Future of Software Engineering, p.175-188, June 04-11, 2000, Limerick, Ireland [doi>10.1145/336512.336551]
• 25: G.J. Hahn and S.S. Shapiro. Statistical Models in Engineering. Wiley & Sons, New York, 1967.
• 26: Bev Littlewood , David Wright, Some Conservative Stopping Rules for the Operational Testing of Safety-Critical Software, IEEE Transactions on Software Engineering, v.23 n.11, p.673-683, November 1997 [doi>10.1109/32.637384]
• 27: Keith W. Miller , Larry J. Morell , Robert E. Noonan , Stephen K. Park , David M. Nicol , Branson W. Murrill , Jeffrey M. Voas, Estimating the Probability of Failure When Testing Reveals No Failures, IEEE Transactions on Software Engineering, v.18 n.1, p.33-43, January 1992 [doi>10.1109/32.120314]
• 28: Joe W. Duran and Simeon C. Ntafos. An evaluation of random testing. IEEE Transactions on Software Engineering, 10(4):438--444, April 1984.
• 29: E.J. Weyuker. Can we measure software testing effectiveness? In Proc. of the First Int'l Software Metrics Symposium, pages 100--107. IEEE, May 1993.
• 30: W. E. Howden, Confidence-Based Reliability And Statistical Coverage Estimation, Proceedings of the Eighth International Symposium on Software Reliability Engineering (ISSRE '97), p.283, November 02-05, 1997
• 31: Giovanni Denaro , Sandro Morasca , Mauro Pezzè, Deriving models of software fault-proneness, Proceedings of the 14th international conference on Software engineering and knowledge engineering, July 15-19, 2002, Ischia, Italy [doi>10.1145/568760.568824]
• 32: Thomas J. Ostrand , Elaine J. Weyuker , Robert M. Bell, Predicting the Location and Number of Faults in Large Software Systems, IEEE Transactions on Software Engineering, v.31 n.4, p.340-355, April 2005 [doi>10.1109/TSE.2005.49]
• 33: V. Okun, P.E. Black, and Y. Yesh. Comparison of fault classes in specification-based testing. Information and Software Technology, 46(5):525--533, November 2004.

INDEX TERMS

Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.8 Metrics
          Subjects: Process metrics; Product metrics

General Terms:
Design, Measurement, Reliability

Keywords:
beta distribution, measure, software testability, testing effectiveness