Quality guidelines = designer metrics

ABSTRACT

In spite of the significant body of research on traditional source code metrics, there has been a general failure to produce conclusive evidence as to their effectiveness for measuring software quality. We describe and recommend a potentially much more powerful and sensitive quality assessment alternative, software quality guidelines. Software quality guidelines are presented as “designer metrics”, that is, user-defined rules or constraints relating to measurable features of a program's structure, semantics, and syntax that affect its quality. To provide a methodology for designing, applying, and validating software quality guidelines, we recommend and briefly summarize IEEE standard 1061. This standard gives a process for constructing and implementing a software quality metrics framework that can be tailor-made to meet quality requirements for a particular project and/or organization. Our paper then demonstrates how software quality guidelines fit within the IEEE framework and gives an example illustrating how user-defined guidelines can be applied to evaluate or assess the quality of an Ada source unit. This guideline-based assessment of quality is then compared with an analysis based on traditional McCabe and Halstead metrics. Finally, we introduce a tool being developed by SAIC, called AdaReVu, as an effective mechanism for implementing and applying user-defined quality guidelines for Ada source code.

REFERENCES

• ART85: Lowell Jay Arthur, Measuring Programmer Productivity and Software Quality, John Wiley & Sons, Inc., New York, NY, 1985
• ARN90: Arnold, Robert S.,"Heuristics for Salvaging Reusable Parts from Ada Source Code", Software Productivity Consortium Technical Report, Ada Reuse Heuristics 9001 I-N, March 1990.
• BAS83: Basili, Victor R., Richard W. Selby and Tsai-Yun Phillips. "Metric Analysis and Data Validation Across Fortran Projects." IEEE Transactions on Software Engineering SE-9. 1983: 652-663.
• BAS84: Victor R. Basili , Barry T. Perricone, Software errors and complexity: an empirical investigation0, Communications of the ACM, v.27 n.1, p.42-52, Jan. 1984 [doi>10.1145/69605.2085]
• BER86: Berard, E.V., "Creating Reusable Ada Software", Proc. Natl. Conf. on Software Reusability and Maintainability, September 10-11, 1986.
• BOW84: John B. Bowen, Are current approaches sufficient for measuring software quality?, Proceedings of the software quality assurance workshop on Functional and performance issues, p.148-155, January 1978
• DeM82: T. DeMarco, Controlling Software Projects: Management, Measurement, and Estimates, Prentice Hall PTR, Upper Saddle River, NJ, 1986
• HEN84: Henry, Sallie and Dennis Kafura. "The Evaluation of Software Systems' Structure Using Quantitative Software Metrics." Software Practice and Experience. Vol. 14 1984: 561-573.
• IEEE93: IEEE, Inc. IEEE Standard for a Software Quality Metrics Methodology. IEEE Computer Society. New York: 1993.
• KEA86: Joseph P. Kearney , Robert L. Sedlmeyer , William B. Thompson , Michael A. Gray , Michael A. Adler, Software complexity measurement, Communications of the ACM, v.29 n.11, p.1044-1050, Nov. 1986 [doi>10.1145/7538.7540]
• LI91: Li, Wei, Sallie Henry, and Calvin Selig. "Measuring Ada Design to Predict Maintainability." 9th Annual National Conference on Ada Technology (1991): 107-113.
• McC77: McCall, J., P. Richards, and G. Waiters, "Factors in Software Quality," Three Volumes, NTIS AD-A049-014, 015,055, Nov., 1977.
• NAU69: Naur, P., and B. Randell (eds.), Software Engineering: Areport on a Conference sponsored by the NATO Science Committee, NATO, 1969.
• PRE92: Roger S. Pressman, Software engineering (3rd ed.): a practitioner's approach, McGraw-Hill, Inc., New York, NY, 1992
• ROM90: H. Dieter Rombach, Design Measurement: Some Lessons Learned, IEEE Software, v.7 n.2, p.17-25, March 1990 [doi>10.1109/52.50770]
• SCH93: Schneidewind, N., "Report on the iEEE Standard for a Software Quality Metrics Methodology", ACM Software Engineering Notes, Vol. 18, No. 3, July 1993.
• SHE88: Martin Shepperd, An evaluation of software product metrics, Information and Software Technology, v.30 n.3, p.177-188, April 1988 [doi>10.1016/0950-5849(88)90064-X]
• SPC92: Software Productivity Consortium. Ada Style and Quality: Guidelines for Professional Programmers. 1992.
• STD86: St. Dennis, Richard, A Guidebook for Writing Reusable Source Code in Ada, Ver 1.1, Honeywell Computer Sciences Center, Technical Report, CSC-86-3, May 1986.
• TRA87: Tracz, W., "Ada Reusability Efforts: A Survey of the State of Practice", Tech. Report, Program Analysis and Verification Group, Computer Systems Laboratory, ERL 402, Stanford University, October 1987.
• WOR93a: Workman, David. "An Approach to implementing Reusability Guidelines for Ada," $oftwarc Technology Conference. 1993.
• WOR93b: Workman, David. "An Overview of AdaReVum,'' SAIC Technical Paper. 1993.

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.9 Management
          Subjects: Software quality assurance (SQA)
  D.3 PROGRAMMING LANGUAGES
      D.3.2 Language Classifications

          Nouns: Ada

General Terms:
Algorithms, Languages, Measurement, Standardization, Verification