Tools and experiments supporting a testing-based theory of component composition

ABSTRACT

Development of software using off-the-shelf components seems to offer a chance for improving product quality and developer productivity. This article reviews a foundational testing-based theory of component composition, describes tools that implement the theory, and presents experiments with functional and nonfunctional component/system properties that validate the theory and illuminate issues in component composition.

The context for this work is an ideal form of Component-Based Software Development (CBSD) supported by tools. Component developers describe their components by measuring approximations to functional and nonfunctional behavior on a finite collection of subdomains. Systems designers describe an application-system structure by the component connections that form it. From measured component descriptions and a system structure, a CAD tool synthesizes the system properties, predicting how the system will behave. The system is not built, nor are any test executions performed. Neither the component sources nor executables are needed by systems designers. From CAD calculations a designer can learn (approximately) anything that could be learned by testing an actual system implementation. The CAD tool is often more efficient than it would be to assemble and execute an actual system.

Using tools that support an ideal separation between component- and system development, experiments were conducted to investigate two related questions: (1) To what extent can unit (that is, component) testing replace system testing? (2) What properties of software and subdomains influence the quality of subdomain testing?

REFERENCES

• 1: Corrado Böhm , Giuseppe Jacopini, Flow diagrams, turing machines and languages with only two formation rules, Communications of the ACM, v.9 n.5, p.366-371, May 1966 [doi>10.1145/355592.365646]
• 2: Greg Dennis , Felix Sheng-Ho Chang , Daniel Jackson, Modular verification of code with SAT, Proceedings of the 2006 international symposium on Software testing and analysis, July 17-20, 2006, Portland, Maine, USA [doi>10.1145/1146238.1146251]
• 3: Michael D. Ernst , Jake Cockrell , William G. Griswold , David Notkin, Dynamically Discovering Likely Program Invariants to Support Program Evolution, IEEE Transactions on Software Engineering, v.27 n.2, p.99-123, February 2001 [doi>10.1109/32.908957]
• 4: Floyd, R. W. 1967. Assigning meanings to programs. In Proceedings of the Symposium on Applied Mathematics. Vol. 19. American Mathematical Society, 19--32.
• 5: Goodenough, J. B. and Gerhart, S. L. 1975. Toward a theory of test data selection. IEEE Trans. Soft. Engin. 1, 156--173.
• 6: Hamlet, D. 1994. Random testing. In Encyclopedia of Software Engineering, J. Marciniak, Ed. Wiley, New York, 970--978.
• 7: Dick Hamlet, Subdomain testing of units and systems with state, Proceedings of the 2006 international symposium on Software testing and analysis, July 17-20, 2006, Portland, Maine, USA [doi>10.1145/1146238.1146249]
• 8: Hamlet, D. 2007a. www.cs.pdx.edu/~hamlet/components.html.
• 9: Dick Hamlet, Software component composition: a subdomain-based testing-theory foundation, Software Testing, Verification & Reliability, v.17 n.4, p.243-269, December 2007 [doi>10.1002/stvr.v17:4]
• 10: Hamlet, D., Andric, M., and Tu, Z. 2003. Experiments with composing component properties. In Proceedings of the 6th ICSE Workshop on Component-Based Software Engineering, K. Wallnau, Ed. http://www.sei.cmu.edu/pacc.
• 11: Dick Hamlet , Dave Mason , Denise Woit, Theory of software reliability based on components, Proceedings of the 23rd International Conference on Software Engineering, p.361-370, May 12-19, 2001, Toronto, Ontario, Canada
• 12: W. E. Howden, Reliability of the Path Analysis Testing Strategy, IEEE Transactions on Software Engineering, v.2 n.3, p.208-215, May 1976 [doi>10.1109/TSE.1976.233816]
• 13: Jeff Magee, Concurrency: State Models And Java Programs, John Wiley & Sons, 2006
• 14: Lions, J. L. 1996. ARIANE 5, Flight 501 Failure, Report by the Inquiry Board. European Space Agency (ESA), Paris.
• 15: Karl Meinke, Automated black-box testing of functional correctness using function approximation, Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis, July 11-14, 2004, Boston, Massachusetts, USA
• 16: Bertrand Meyer, Object-oriented software construction (2nd ed.), Prentice-Hall, Inc., Upper Saddle River, NJ, 1997
• 17: Harlan D. Mills, Princples of Computer Programming, Allyn & Bacon, Inc., Needham Heights, MA, 1987
• 18: John Musa, Software Reliability Engineered Testing, McGraw-Hill, Inc., New York, NY, 1998
• 19: Glenford J. Myers, Art of Software Testing, John Wiley & Sons, Inc., New York, NY, 1979
• 20: Xie, F. and Browne, J. 2006. Verification of component-based software application families. Lecture Notes in Computer Science, vol. 4063, I. Gorton et al., Eds. Springer, 50--66.

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging

General Terms:
Verification

Keywords:
CAD tool support for CBSD, Experiments with composition of software components, component-based software development (CBSD), synthesis of system properties