Goal-oriented software assessment

ABSTRACT

Companies that engage in multi-site, multi-project software development continually face the problem of how to understand and improve their software development capabilities. We have defined and applied a goal-oriented process that enables such a company to assess the strengths and weaknesses of those capabilities. Our goals are to help a) to decrease the time and cost to develop software, b)to decrease the time needed to make changes to existing software, c) to improve software quality, d) to attract and retain a talented engineering staff, and e) to facilitate more predictable management of software projects. In response to the variety of product requirements, market needs, and development environments, we selected a goal-oriented process, rather than a criteria-oriented process, to advance our strategy and ensure relevance of the results. We describe the design of the process, discuss results achieved, and present vulnerabilities of the methodology. The process includes both interviews with projects' personnel and analysis of change data. Several common issues have emerged from the assessments across multiple projects, enabling strategic investments in software technology. Teams report satisfaction with the outcome in that they act on the recommendations, ask for additional future assessments, and recommend the process to sibling organizations.

REFERENCES

• 1: Basili, V.R., Weiss, D., "A Methodology for Collecting Valid Software Engineering Data," IEEE Transactions on Software Engineering, Vol. 10, No. 3, November 1984, pp. 728-738.
• 2: Victor R. Basili and H. Dieter Rombach. "The TAME Project: Towards Improvement-Oriented Software Environments", IEEE Transactions on Software Engineering, v. SE-14, n. 6, June 1988, pp. 758-773.
• 3: Victor Basili , Scott Green, Software Process Evolution at the SEL, IEEE Software, v.11 n.4, p.58-66, July 1994 [doi>10.1109/52.300090]
• 4: Birk, A., Hamman, D., Pfahl, D., Järvinen, J., Oivo, M., Vierimaa, M., van Solingen, R., The Role of GQM in the PROFES Improvement Methodology, CONQUEST 99.
• 5: Barry W. Boehm, Software Engineering Economics, Prentice Hall PTR, Upper Saddle River, NJ, 1981
• 6: Dalkey, Norman C., Delphi, P-3704., The Rand Corporation, Santa Monica, CA 1967.
• 7: Dalkey, N., Brown, Cochran, The Delphi Method, III: Use of Self Ratings to Improve Group Estimates, RAND, RM-6115-PR, 1969. Can be found in www.rand.org/publications/classics/delphi3.pdf
• 8: Stephen G. Eick , Todd L. Graves , Alan F. Karr , J. S. Marron , Audris Mockus, Does Code Decay? Assessing the Evidence from Change Management Data, IEEE Transactions on Software Engineering, v.27 n.1, p.1-12, January 2001 [doi>10.1109/32.895984]
• 9: Robert B. Grady , Deborah L. Caswell, Software metrics: establishing a company-wide program, Prentice-Hall, Inc., Upper Saddle River, NJ, 1987
• 10: Grady, Robert B. and D. Caswell, "Understanding HP's Software Development Processes through Software Metrics," HP Software Productivity Conference Procedings (April 1984), pp 3-38- 3-54.
• 11: Jones, C., "Measuring Programming Quality and Productivity," IBM Systems Journal, Vol. 17, no. 1 (1978), pp. 39-63.
• 12: C. Jones, Tutorial programming productivity: issues for the eighties, 2nd ed., IEEE Computer Society Press, Los Alamitos, CA, 1986
• 13: Capers Jones, Software assessments, benchmarks, and best practices, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 2000
• 14: Kimball, Ralph, The Data Warehouse Toolkit, Wiley, 1996.
• 15: Ralph Kimball , Laura Reeves , Warren Thornthwaite , Margy Ross , Warren Thornwaite, The Data Warehouse Lifecycle Toolkit: Expert Methods for Designing, Developing and Deploying Data Warehouses with CD Rom, John Wiley & Sons, Inc., New York, NY, 1998
• 16: Bennett P. Lientz , E. Burton Swanson, Software Maintenance Management, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1980
• 17: Linstone, Harold A., and Murray Turoff, eds. Delphi Method: Techniques and Applications. Reading, MA, Addison-Wesley Publishing Company, 1975.
• 18: Mockus, A., Eick, S.G., Graves, T. L., and Karr, A. F., "On Measurement and Analysis of Software Changes," Tech. Rep., National Institute of Statistical Sciences, 1999.
• 19: Paulk, M.C., Curtis, B., Chrissis, M.B., et. al., Capability Maturity Model for Software, Version 1.1, Software Engineering Institute, CMU/SEI-93-TR-25, DTIC Number ADA263432, February 1993.
• 20: Turoff, M., and Hiltz, Starr Roxanne, Computer Based Delphi Processes, http://eies.njit.edu/~turoff/Papers/delphi3.html.
• 21: Shewhart, Walter A., Statistical Method from the Viewpoint of Quality Control, Graduate School, Department of Agriculture, Washington 1939; Dover 1986.
• 22: Walston, C.E., and C.P. Felix, "A Method of Programming Measurement and Estimation," IBM Systems Journal, no. 1 (1977), pp. 54-73.
• 23: David M. Weiss , Chi Tau Robert Lai, Software product-line engineering: a family-based software development process, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1999
• 24: Weiss, D.M. and V.R. Basili, "Evaluating Software Development by Analysis of Changes: Some Data from the Software Engineering Laboratory," IEEE Transactions on Software Engineering Vol. SE-11, No. 2 (Feb 1985), pp. 157-168.

INDEX TERMS

Primary Classification:
  K. Computing Milieux
  K.6 MANAGEMENT OF COMPUTING AND INFORMATION SYSTEMS
      K.6.3 Software Management
          Subjects: Software development

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.1 Requirements/Specifications
          Subjects: Methodologies (e.g., object-oriented, structured)
      D.2.9 Management
          Subjects: Software configuration management

General Terms:
Algorithms, Design, Performance