System structure and software maintenance performance

ABSTRACT

An experiment is designed to investigate the relationship between system structure and maintainability. An old, ill-structured system is improved in two sequential stages, yielding three system versions for the study. The primary objectives of the research are to determine how or whether the differences in the systems influence maintenance performance; whether the differences are discernible to programmers; and whether the differences are measurable. Experienced programmers perform a portfolio of maintenance tasks on the systems. Results indicate that system improvements lead to decreased total maintenance time and decreased frequency of ripple effect errors. This suggests that improving old systems may be worthwhile and may yield benefits over the remaining life of the system. System differences are not discernible to programmers; apparently programmers are unable to separate the complexity of the systems from the complexity of the maintenance tasks. This finding suggests a need for further research on the efficacy of subjectively based software metrics. Finally, results indicate that a selected set of automatable, objective complexity metrics reflected both the improvements in the system and programmer maintenance performance. These metrics appear to offer potential as project management tools.

REFERENCES

• 1: Agresti, W.W. Managing program maintenance. J. Syst. Manage. 33, 2 (Feb. 1982), 34-37.
• 2: Agresti, W.W. Measuring program maintainability. J. Syst. Manage. 33, 3 (Mar. 1982), 26-29.
• 3: Jack J. Baroudi , Michael J. Ginzberg, Impact of the technological environment on programmer/analyst job outcomes, Communications of the ACM, v.29 n.6, p.546-555, June 1986 [doi>10.1145/5948.5953]
• 4: Basili, V.R. Tutorial on Models and Metrics for Software Management and Engineering. IEEE, New York, 1980.
• 5: Basili, V.R., and Reiter, R.W. Evaluating automatable measures of software development. In Workshop on Quantitative Software Models for Reliability, Complexity, and Cost. IEEE, New York, 1979, pp. 107-116.
• 6: V R Basili , R W Selby , D H Hutchens, Experimentation in software engineering, IEEE Transactions on Software Engineering, v.12 n.7, p.733-743, July 1986
• 7: Benyon-Tinker, G. Complexity measures in an evolving large system. In Workshop on Quantitative Software Models. IEEE, New York, 1979, pp. 117-127.
• 8: Gerald M. Berns, Assessing software maintainability, Communications of the ACM, v.27 n.1, p.14-23, Jan. 1984 [doi>10.1145/69605.357965]
• 9: Boehm, B.W. Software engineering economics. IEEE Trans. Softw. Eng. 10, I (Jan. 1984}, 4-21.
• 10: William Bryan , Stanley Y. Siegel, Making software visible, operational, and maintainable in a small project environment, IEEE Transactions on Software Engineering, v.10 n.1, p.59-67, January 1984 [doi>10.1109/TSE.1984.5010199]
• 11: Canning, R. Easi:lg the software maintenance burden. EDP Analyzer 19, 8 (Aug. 1981) 1-13.
• 12: Chen, E.T. Progr~lm complexity and programmer productivity. IEEE Trans. Sofiw. Eng. 4, 3 (May 1978), 187-194.
• 13: Clapp, J.A. Desig~fing software for maintainability. Computer Design 20, 9 (Sept. 1981) 197-202.
• 14: Curtis, B. et al. Iv easuring the psychological complexity of software maintenance tasls with the Halstead and McCabe metrics. IEEE Trans. Softw. Eng. 5, 2 (Mar. 1979), 96-104.
• 15: Bill Curtis , Sylvia B. Sheppard , Phil Milliman, Third time charm: Stronger prediction of programmer performance by software complexity metrics, Proceedings of the 4th international conference on Software engineering, p.356-360, September 17-19, 1979, Munich, Germany
• 16: Peter J. Denning, ACM president's letter: throwaway programs, Communications of the ACM, v.24 n.2, p.57-58, Feb. 1981 [doi>10.1145/358549.358554]
• 17: Edsger W. Dijkstra, How do we tell truths that might hurt?, ACM SIGPLAN Notices, v.17 n.5, p.13-15, May 1982 [doi>10.1145/947923.947924]
• 18: Edwards, C. Infer:nation systems maintenance: An integrated perspective. MIS Quarterly 8, 4 (Dec. 1984), 237-256.
• 19: James L. Elshoff , Michael Marcotty, Improving computer program readability to aid modification, Communications of the ACM, v.25 n.8, p.512-521, Aug 1982 [doi>10.1145/358589.358596]
• 20: Michael Evans, Software engineering for the Cobol environment, Communications of the ACM, v.25 n.12, p.874-882, Dec 1982 [doi>10.1145/358728.358732]
• 21: Federal agencies' naintenance of computer programs: Expensive and undermanage :1, GAO Report to Congress AFMD-81-25, Feb. 26, 1981.
• 22: Gaffney, }.E., Jr. P~ogram control complexity and productivity. In Workshop on Quan itative Software Models, IEEE, New York, 1979.
• 23: Virginia Rae Gibson, A study of complexity metrics as surrogate measures of software maintainability, State University of New York at Binghamton, Binghamton, NY, 1986
• 24: Glass, R.L., and Nciseux, R.A. Software maintenance guidebook. Prentice-Hall, Eng ewood Cliffs, NJ, 1981.
• 25: Lee L. Gremillion, Determinants of program repair maintenance requirements, Communications of the ACM, v.27 n.8, p.826-832, Aug 1984 [doi>10.1145/358198.358228]
• 26: Guidance on softwa~'e maintenance, National Bureau of Standards Publication no. 500-1013, U.S. Government Printing Office, Washington, D.C.. 1983.
• 27: Maurice H. Halstead, Elements of Software Science (Operating and programming systems series), Elsevier Science Inc., New York, NY, 1977
• 28: Harrison, W. Struclured programming with COBOL. J. Syst. Manage. 32, 3 (Mar. 1981), 13-20.
• 29: Harrison, W., et al. Applying software complexity metrics to program maintenance. Computer 15, 9 (Sept. 1982), 65-79.
• 30: Kirk, R.E. Experime~tal design, Wadsworth Publishing Company, Belmont, CA, 1968.
• 31: Bennett P. Lientz , E. Burton Swanson, Software Maintenance Management, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1980
• 32: Bennet P. Lientz , E. Burton Swanson, Problems in application software maintenance, Communications of the ACM, v.24 n.11, p.763-769, Nov. 1981 [doi>10.1145/358790.358796]
• 33: Martin, J., and McC{ure, C.L. Maintenance of computer software. Savant Institute Rel~ort, Carnforth, England, March 1982.
• 34: McCabe, T.J, A complexity measure. IEEE Trans. Softw. Eng. 2, 4 (Dec. 1976}, 308-32{).
• 35: Carma L. McClure, Managing Software Development and Maintenance, John Wiley & Sons, Inc., New York, NY, 1981
• 36: Parikh, G. Tutorial cn software maintenance. IEEE, New York, 1983.
• 37: Peercy, D.E. A sofm are maintainability evaluation methodology. IEEE Trans. Softw. E~zg. 7, 4 (July 1981), 343-351.
• 38: Pennington, N. Stimulus structures and mental representations in expert comprehensi )n of computer programs. Cogn. Psych. 19, 3 (July 1987), 295-341.
• 39: Rittersbach, G. Technology. World (Peat Marwick}, (Jam-Feb. 1986), 5.
• 40: Svend Ryge, Evaluating structured COBOL as a software engineering discipline, ACM SIGMIS Database, v.12 n.3, p.3-6, Spring 1981 [doi>10.1145/1017674.1017675]
• 41: Sheppard, S.B., Milliman, P., and Love, T. Modern coding practices and programmer performance. Computer 12, 12 (Dec. 1979), 41-49.
• 42: Allen van Gelder, Structured programming in Cobol: an approach for application programmers, Communications of the ACM, v.20 n.1, p.2-12, Jan. 1977 [doi>10.1145/359367.359368]
• 43: Iris Vessey , Ron Weber, Some factors affecting program repair maintenance: an empirical study, Communications of the ACM, v.26 n.2, p.128-134, Feb. 1983 [doi>10.1145/358024.358057]
• 44: Vessey, I., and Weber, R, Research on structured programming: An empiricist's evaluation. IEEE Trans. Softw. Eng. 10, 4 (July 1984), 397-407.
• 45: Weiner-Ehrlich, W.K., Hamrick, J.R., and Rupolo, V.F. Modeling software behavior in terms of a formal life cycle curve: Implications for software maintenance. IEEE Trans. Softw. Eng. 10, 4 (July 1984}, 376-383.
• 46: Mark Weiser, Programmers use slices when debugging, Communications of the ACM, v.25 n.7, p.446-452, July 1982 [doi>10.1145/358557.358577]
• 47: Weiss, D.M., and Basili, V.R. Evaluating software development by analysis of changes: Some data from the software engineering laboratory. IEEE Trans. Softw. Eng. 11, 2 (Feb. 1985}, 157-168.
• 48: Winer, B.J. Statistical principles in experimental design, McGraw-Hill, New York, 1971.
• 49: John R. Wolberg, Comparing the cost of software conversion to the cost of reprogramming, ACM SIGPLAN Notices, v.16 n.4, April 1981 [doi>10.1145/988131.988144]
• 50: Woodward, M.R., Hennell. M.A., and Hedley, D. A measure of control flow complexity in program text. IEEE Trans. Softw. Eng. 5, 1 (Jan. 1979), 45-50.
• 51: You, S.S., and Collofello, J.S. Some stability measures for software maintenance. IEEE Trans. Softw. Eng. 6, 6 (Nov. 1980), 545-552.
• 52: Zolnowski, J.C., and Simmons, D.B. Taking the measure of program complexity. Prec. AFIPS, 1981, NCC, 329-336,

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.8 Metrics
          Subjects: Performance measures

General Terms:
Experimentation, Measurement, Performance