A case study in test management

ABSTRACT

Testing is an essential but often under-utilized area of software engineering. A variety of software testing techniques have been developed to effectively identify bugs in source code, yet these techniques are not always fully employed in practice. There are numerous reasons for this, including the difficulty in mastering the complexity of managing all of the test cases for large-scale projects. Test case management involves organizing testing artifacts (e.g., requirements traceability data, test cases, and expected results) in a systematic manner. To be successful, test case management requires a high degree of discipline to accommodate the large volume of artifacts under consideration. This paper presents the results of a case study in centralizing test artifacts in an industrial setting to aid better test management. Several of the challenges in adopting this approach are discussed. In response to these challenges, recommendations on how to better leverage test case management are offered.

REFERENCES

• 1: William Perry, Effective methods for software testing, Wiley-QED Publishing, Somerset, NJ, 1995
• 2: Mary Jean Harrold, Testing: a roadmap, Proceedings of the Conference on The Future of Software Engineering, p.61-72, June 04-11, 2000, Limerick, Ireland [doi>10.1145/336512.336532]
• 3: Cem Kaner , James Bach , Bret Pettichord, Lessons Learned in Software Testing, John Wiley & Sons, Inc., New York, NY, 2001
• 4: Mercury, http://www.mercury.com/. Last accessed - November 22, 2006.
• 5: Keith Miller , Jeffrey Voas, Software Test Cases: Is One Ever Enough?, IT Professional, v.8 n.1, p.44-48, January 2006 [doi>10.1109/MITP.2006.28]
• 6: Cem Kaner , Jack L. Falk , Hung Quoc Nguyen, Testing Computer Software, Second Edition, John Wiley & Sons, Inc., New York, NY, 1999
• 7: Glenford J. Myers, Art of Software Testing, John Wiley & Sons, Inc., New York, NY, 1979
• 8: James A. Whittaker, How to Break Software: A Practical Guide to Testing with Cdrom, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 2002
• 9: Kaner, C. Architectures of Test Automation. Software Testing, Analysis & Review Conference (Star) West, San Jose, CA, Oct 2000.
• 10: B. Korel, Automated Software Test Data Generation, IEEE Transactions on Software Engineering, v.16 n.8, p.870-879, August 1990 [doi>10.1109/32.57624]
• 11: Kaner, C. "Improving the Maintainability of Automated Test Suites", Software QA, 4(4), 1997.
• 12: Beck, Test Driven Development: By Example, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 2002

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.9 Management

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.2 Automatic Programming
          Subjects: Program verification

  K. Computing Milieux
  K.6 MANAGEMENT OF COMPUTING AND INFORMATION SYSTEMS
      K.6.3 Software Management

Keywords:
software testing, test management