Regression test selection on system requirements

ABSTRACT

Regression testing, which is performed after changes are made to a software system, can be used before release of new versions of the system. However, practitioners often have little time to perform this regression testing because of the quick-release cycles of such modified systems. Thus, they may use a random-testing approach or perform little regression testing. This lack of adequate regression testing can cause bugs in untested parts of the program to be exposed only during production or field usage. To improve the efficiency of the regression testing, and thus enable its use before release, techniques that select and run only those test cases that are related to the changes or prioritize the test cases based on criticality or perceived effectiveness have been presented. These technique typically use some representation of the software such as a system model or the source code to perform the test selection and prioritization. However, in practice, access to a system modelor the source code may not be possible. To provide regression test selection in practice, we have developed, and present in this paper, a novel approach to regression test selection that, instead of using a system model or the source code, uses the system requirements and their associated test cases, which are typically available to developers/testers. The approach uses the set of system requirements, usually in natural language or some informal notation, that represent what is to be tested about the system. The technique uses these requirements, along with the set of test cases and their criticality that are associated with the requirements, to select test cases for use in regression testing. In this paper, we also present a case study that shows the potential effectiveness of our technique

REFERENCES

• 1: Tata Consultancy Services Limited. http://www.tcs.com/AboutUs/AboutUs.html.
• 2: Thomas Ball, On the limit of control flow analysis for regression test selection, Proceedings of the 1998 ACM SIGSOFT international symposium on Software testing and analysis, p.134-142, March 02-04, 1998, Clearwater Beach, Florida, United States
• 3: Boris Beizer, Software testing techniques (2nd ed.), Van Nostrand Reinhold Co., New York, NY, 1990
• 4: Yih-Farn Chen , David S. Rosenblum , Kiem-Phong Vo, TestTube: a system for selective regression testing, Proceedings of the 16th international conference on Software engineering, p.211-220, May 16-21, 1994, Sorrento, Italy
• 5: P. K. Chittimalli, M. Bapat, and R. D. Naik. ProAX: A program analysis and transformation framework. In TCS Technical Architect's Conference (TACTiCS 2004), Hyderabad, December 2004.
• 6: P. K. Chittimalli and M. J. Harrold. Re-computing coverage information to assist regression testing. In International Conference on Software Maintenance (ICSM 2007), pages 164--173, October 2007.
• 7: D. Kung, J. Gao, P. Hsia, Y. Toyashima, and C. Chen. Firewall regression testing and software maintenance of object-oriented systems. 1994.
• 8: H. K. N. Leung and L. J. White. Insights into regression testing. In Proceedings of the Conference on Software Maintenance -- 1989, pages 60--69, 1989.
• 9: Alessandro Orso , Nanjuan Shi , Mary Jean Harrold, Scaling regression testing to large software systems, Proceedings of the 12th ACM SIGSOFT twelfth international symposium on Foundations of software engineering, October 31-November 06, 2004, Newport Beach, CA, USA
• 10: Gregg Rothermel , Mary Jean Harrold, A safe, efficient regression test selection technique, ACM Transactions on Software Engineering and Methodology (TOSEM), v.6 n.2, p.173-210, April 1997 [doi>10.1145/248233.248262]
• 11: Gregg Rothermel , Mary Jean Harrold, Analyzing Regression Test Selection Techniques, IEEE Transactions on Software Engineering, v.22 n.8, p.529-551, August 1996 [doi>10.1109/32.536955]
• 12: G. Rothermel, R. Untch, C. Chu, and M. J. Harrold. Test case prioritization. In Technical Report GIT-99-28, College of Computing, Georgia Institute of Technology (GIT-99-28 1999), Georgia Institute of Technology, December 1999.
• 13: Gregg Rothermel , Roland H. Untch , Chengyun Chu , Mary Jean Harrold, Test Case Prioritization: An Empirical Study, Proceedings of the IEEE International Conference on Software Maintenance, p.179, August 30-September 03, 1999
• 14: F. I. Vokolos , P. G. Frankl, Pythia: a regression test selection tool based on textual differencing, IFIP TC5 WG5.4 3rd internatinal conference on on Reliability, quality and safety of software-intensive systems, p.3-21, January 1997, Athens, Greece

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging
          Subjects: Testing tools (e.g., data generators, coverage testing)

General Terms:
Experimentation

Keywords:
regression test selection, regression testing, requirements, test case prioritization, traceability