An empirical validation of software cost estimation models

ABSTRACT

Practitioners have expressed concern over their inability to accurately estimate costs associated with software development. This concern has become even more pressing as costs associated with development continue to increase. As a result, considerable research attention is now directed at gaining a better understanding of the software-development process as well as constructing and evaluating software cost estimating tools. This paper evaluates four of the most popular algorithmic models used to estimate software costs (SLIM, COCOMO, Function Points, and ESTIMACS). Data on 15 large completed business data-processing projects were collected and used to test the accuracy of the models' ex post effort estimation. One important result was that Albrecht's Function Points effort estimation model was validated by the independent data provided in this study [3]. The models not developed in business data-processing environments showed significant need for calibration. As models of the software-development process, all of the models tested failed to sufficiently reflect the underlying factors affecting productivity. Further research will be required to develop understanding in this area.

REFERENCES

• 1: Abdel-Hamid. T.. and Madnick, S. Impact of schedule estimation on software project behavior. IEEE Softw. 3,4 (July 1986), 70-75.
• 2: Albrecht, A.J. Measuring application development productivity. In Proceedinns of fhe IBM Applications Development Symposium, GUIDE/ SHARE (Goiterey, Calic,'Oct. 14-17). IBM, 1979,'pp. 83-92.
• 3: Albrecht. A.J.. and Gaffney. J., Jr. Software function, source lines of code, and development effort prediction: A software science validation. IEEE Trans. Softw. Eng. SE-g, 6 (Nov. 1963), 639-646.
• 4: Behrens. CA. Measuring the productivity of computer systems development activities with Function Points. IEEE Trans. Softw. Eng. SE-g, 6 (Nov. 1963). 646-652.
• 5: Barry W. Boehm, Software Engineering Economics, Prentice Hall PTR, Upper Saddle River, NJ, 1981
• 6: Frederick P. Brooks, Jr., The Mythical Man-Month: Essays on Softw, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1978
• 7: Call&m, H., and Colborne, S. A proposed method for estimating software cost from requirements. J. Parmetrics 4,4 (Dec. 1984), 33-40.
• 8: S. D. Conte , H. E. Dunsmore , V. Y. Shen, Software engineering metrics and models, Benjamin-Cummings Publishing Co., Inc., Redwood City, CA, 1986
• 9: Ferens, D.V. Software support cost models: Quo vadis? J. Pammetrics 4. 4 (Dec. 1984). 64-99.
• 10: Gaffney. J.E.. Goldberg. R.. and Misek-Falkoff. L. SCORE82 Summary. Perform. Eval. Rev. 12.4 (Winter 1984-1985). 4-12.
• 11: John R. Golden , James R. Mueller , Barbara Anselm, Software cost estimating: craft or witchcraft, ACM SIGMIS Database, v.12 n.3, p.12-14, Spring 1981 [doi>10.1145/1017674.1017677]
• 12: Jones, C. Programming Productivity. McGraw-Hill, New York, 1986.
• 13: Kitchenham. B.. and Taylor, N.R. Software cost models. ICL Tech. J. 4.1 (May 1984), 73-102.
• 14: Masters, T.F.. II. An overview of software cost estimating at the NSA. J Parmetrics 5. 1 (Mar. 1985), 72-84.
• 15: Pinsky, S.S. The effect of complexity on software trade off equations. J, Pammetrics 4, 4 (Dec. 1984), 23-32.
• 16: Putnam. L.H. General empirical solution to the macro software sizing and estimating problem. IEEE Trans. Soffw. Eng. SE 4, 4 (July 19781, 345-361.
• 17: Putnam. L.. and Fitzsimmons, A. Estimating software costs. Datamation 25, lo-12 (Sept.-Nov. 1979).
• 18: Quantitative Software Management. Refprence Notes for the DOD SLIM Sofrware Cost Estimating Course. Quantitative Software Management. McLean. Va., 1983.
• 19: Quantitative Software Management. SLIM User Manual (IBM PC Version) Draft copy ed. Quantitative Software Management. McLean, Va.. 1984.
• 20: Rubin, H.A. Macroestimation of software development parameters: The Estimacs system. In SOFTFAIR Conference on Software Deuelopment Took, Techniques and AIternatiues (Arlington, Va., July 25-28). IEEE Press, New York, 1983. pp. 109-118.
• 21: Rubin. H.A. The art and science of software estimation: Fifth generation estimators. In Proceedings of the 7th Annual ISPA Conference (Orlando, Fla., May 7-9). International Society of Parametric Analysts, McLean, Va.. 1985. pp. 56-72.
• 22: Robin, H.A. Using ESTIMACS E. Management and Computer Services, Valley Forge, Pa., Mar. 1984.
• 23: Software Productivity Research. SPQR/ZO User Guide. Software Productivity Research, Cambridge, Mass., 1986.
• 24: Theabaut. SM. Model evaluation in software metrics research. In Computer Science and Statistics: Proceedings of the 15th Symposium on the Interface (Houston, Tex., Mar.). 1983. pp. 277-285.
• 25: Walston. C.E.. and Felix, C.P. A method of programming measurement and estimation. IBM Syst. J, 26, 1 (Jan. 1977), 54-73.
• 26: W&berg. S. Applied Linear Regression. Wiley, New York, 1980.
• 27: Wolverton, W.R. Cost of developing large scale software. lEEE Trans. Comput. C-23, 6 (June 19741, 615-634.

INDEX TERMS

Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.8 Metrics
          Subjects: Performance measures
      D.2.9 Management
          Subjects: Cost estimation

  K. Computing Milieux
  K.6 MANAGEMENT OF COMPUTING AND INFORMATION SYSTEMS
      K.6.1 Project and People Management
          Subjects: Management techniques (e.g., PERT/CPM)
      K.6.3 Software Management
          Subjects: Software development

General Terms:
Management, Measurement, Performance