A comparison of case-based reasoning approaches

ABSTRACT

Over the years software engineering researchers have suggested numerous techniques for estimating development effort. These techniques have been classified mainly as algorithmic, machine learning and expert judgement. Several studies have compared the prediction accuracy of those techniques, with emphasis placed on linear regression, stepwise regression, and Case-based Reasoning (CBR). To date no converging results have been obtained and we believe they may be influenced by the use of the same CBR configuration.The objective of this paper is twofold. First, to describe the application of case-based reasoning for estimating the effort for developing Web hypermedia applications. Second, comparing the prediction accuracy of different CBR configurations, using two Web hypermedia datasets.Results show that for both datasets the best estimations were obtained with weighted Euclidean distance, using either one analogy (dataset 1) or 3 analogies (dataset 2). We suggest therefore that case-based reasoning is a candidate technique for effort estimation and, with the aid of an automated environment, can be applied to Web hypermedia development effort prediction.

REFERENCES

• 1: Albrecht, J., and Gaffney, J. R., Software function, source lines of code, and development effort prediction: a software science validation, IEEE Transactions on Software Engineering, 9, 6, 639-648, 1983.
• 2: L. Angelis , I. Stamelos, A Simulation Tool for Efficient Analogy Based Cost Estimation, Empirical Software Engineering, v.5 n.1, p.35-68, March 2000 [doi>10.1023/A:1009897800559]
• 3: Tomás Isakowitz , Edward A. Stohr , P. Balasubramanian, RMM: a methodology for structured hypermedia design, Communications of the ACM, v.38 n.8, p.34-44, Aug. 1995 [doi>10.1145/208344.208346]
• 4: Ralph Bergmann, Highlights of the European INRECA Projects, Proceedings of the 4th International Conference on Case-Based Reasoning: Case-Based Reasoning Research and Development, p.1-15, August 02, 2001
• 5: Barry W. Boehm, Software Engineering Economics, Prentice Hall PTR, Upper Saddle River, NJ, 1981
• 6: Deborah A. Boehm-Davis , Lyle S. Ross, Program design methodologies and the software development process, International Journal of Man-Machine Studies, v.36 n.1, p.1-19, Jan. 1992 [doi>10.1016/0020-7373(92)90050-U]
• 7: Grady Booch , James Rumbaugh , Ivar Jacobson, The Unified Modeling Language user guide, Addison Wesley Longman Publishing Co., Inc., Redwood City, CA, 1999
• 8: Lionel C. Briand , Khaled El Emam , Dagmar Surmann , Isabella Wieczorek , Katrina D. Maxwell, An assessment and comparison of common software cost estimation modeling techniques, Proceedings of the 21st international conference on Software engineering, p.313-322, May 16-22, 1999, Los Angeles, California, United States [doi>10.1145/302405.302647]
• 9: Lionel C. Briand , Tristen Langley , Isabella Wieczorek, A replicated assessment and comparison of common software cost modeling techniques, Proceedings of the 22nd international conference on Software engineering, p.377-386, June 04-11, 2000, Limerick, Ireland [doi>10.1145/337180.337223]
• 10: Christodoulou, S. P., Zafiris, P. A., Papatheodorou, T. S., WWW2000: The Developer's view and a practitioner's approach to Web Engineering Proc. Second ICSE Workshop on Web Engineering, 4 and 5 June 2000; Limerick, Ireland, 75-92, 2000.
• 11: Francesco Coda , Carlo Ghezzi , Giovanni Vigna , Franca Garzotto, Towards a Software Engineering Approach to Web Site Development, Proceedings of the 9th international workshop on Software specification and design, p.8, April 16-18, 1998
• 12: S. D. Conte , H. E. Dunsmore , Y. E. Shen, Software engineering metrics and models, Benjamin-Cummings Publishing Co., Inc., Redwood City, CA, 1986
• 13: Counsell, S., Swift, S., Tucker, A., and Mendes, E. An Empirical Investigation of Fault Seeding in Requirements Documents. Proceedings of EASE'01 - Fifth International Conference on Empirical Assessment and Evaluation in Software Engineering, Keele University, Staffordshire, UK, April, 2001.
• 14: A. J. C. Cowderoy , A. J. M. Donaldson , J. O. Jenkins, A Metrics Framework for Multimedia Creation, Proceedings of the 5th International Symposium on Software Metrics, p.144, March 20-21, 1998
• 15: Cowderoy, A.J.C. Measures of size and complexity for web-site content, Proceedings of the Combined 11th European Software Control and Metrics Conference and the 3rd SCOPE conference on Software Product Quality, Munich, Germany, 423-431, 2000.
• 16: DeMarco, T., Controlling Software Projects: Management, Measurement and Estimation, Yourdon: New York, 1982.
• 17: Rachel Fewster , Emilia Mendes, Measurement, Prediction and Risk Analysis for Web Applications, Proceedings of the 7th International Symposium on Software Metrics, p.338, April 04-06, 2001
• 18: Roy T. Fielding , Richard N. Taylor, Principled design of the modern Web architecture, Proceedings of the 22nd international conference on Software engineering, p.407-416, June 04-11, 2000, Limerick, Ireland [doi>10.1145/337180.337228]
• 19: G. R. Finnie , G. E. Wittig , J.-M. Desharnais, A comparison of software effort estimation techniques: using function points with neural networks, case-based reasoning and regression models, Journal of Systems and Software, v.39 n.3, p.281-289, Dec. 1997 [doi>10.1016/S0164-1212(97)00055-1]
• 20: Franca Garzotto , Paolo Paolini , Daniel Schwabe, HDM—a model-based approach to hypertext application design, ACM Transactions on Information Systems (TOIS), v.11 n.1, p.1-26, Jan. 1993 [doi>10.1145/151480.151483]
• 21: Gray, A. and MacDonell, S., Applications of Fuzzy Logic to Software Metric Models for Development Effort Estimation, Proceedings of the 1997 Annual Meeting of the North American Fuzzy Information Processing Society - NAFIPS, Syracuse NY, USA, IEEE, 394-399.
• 22: Gray, A.R., and MacDonell, S.G. A comparison of model building techniques to develop predictive equations for software metrics. Information and Software Technology, 39, 425-437, 1997.
• 23: Andrew R. Gray , Stephen G. MacDonell , Martin J. Shepperd, Factors Systematically Associated with Errors in Subjective Estimates of Software Development Effort: The Stability of Expert Judgment, Proceedings of the 6th International Symposium on Software Metrics, p.216, November 04-06, 1999
• 24: Hughes, R.T., An Empirical investigation into the estimation of software development effort, PhD thesis, Dept. of Computing, the University of Brighton, UK, 1997.
• 25: Jeffery, R., Ruhe, M., and Wieczorek, I., A Comparative study of two software development cost modeling techniques using multi-organizational and company-specific data, Information and Software Technology, 42, 1009-1016, 2000.
• 26: Ross Jeffery , Melanie Ruhe , Isabella Wieczorek, Using Public Domain Metrics To Estimate Software Development Effort, Proceedings of the 7th International Symposium on Software Metrics, p.16, April 04-06, 2001
• 27: Gada F. Kadoda , Michelle Cartwright , Martin J. Shepperd, Issues on the Effective Use of CBR Technology for Software Project Prediction, Proceedings of the 4th International Conference on Case-Based Reasoning: Case-Based Reasoning Research and Development, p.276-290, August 02, 2001
• 28: Kadoda, G., Cartwright, M., Chen, L., and Shepperd, M.J., Experiences Using Case-Based Reasoning to Predict Software Project Effort, Proceedings of the EASE 2000 Conference, Keele, UK, 2000.
• 29: Chris F Kemerer, An empirical validation of software cost estimation models, Communications of the ACM, v.30 n.5, p.416-429, May 1987 [doi>10.1145/22899.22906]
• 30: Kitchenham, B.A., Pickard, L.M., MacDonell, S.G., Shepperd, M.J., What accuracy statistics really measure, IEE Proceedings - Software Engineering, (June 2001), 148, 3.
• 31: Kumar, S., Krishna, B.A., and Satsangi, P.S. Fuzzy systems and neural networks in software engineering project management. Journal of Applied Intelligence, 4, 31-52, 1994.
• 32: David Lowe, Hypermedia and the Web: An Engineering Approach, John Wiley & Sons, Inc., New York, NY, 1999
• 33: Emilia Mendes , Steve Counsell , Nile Mosley, Measurement and Effort Prediction for Web Applications, Web Engineering, Software Engineering and Web Application Development, p.295-310, January 2001
• 34: Emilia Mendes , Steve Counsell , Nile Mosley, Towards the prediction of development effort for hypermedia applications, Proceedings of the twelfth ACM conference on Hypertext and Hypermedia, August 14-18, 2001, Århus, none, Denmark [doi>10.1145/504216.504278]
• 35: Emilia Mendes , Nile Mosley , Steve Counsell, Web Metrics— Estimating Design and Authoring Effort, IEEE MultiMedia, v.8 n.1, p.50-57, January 2001 [doi>10.1109/93.923953]
• 36: Mendes, E., Mosley, N., and Counsell, S., A Comparison of Length, Complexity & Functionality as Size Measures for Predicting Web Design & Authoring Effort, Proceedings of the 2001 EASE Conference, Keele, UK, 1-14, 2001.
• 37: Michau, F., Gentil, S., Barrault, M. Expected benefits of web-based learning for engineering education: examples in control engineering. European Journal of Engineering Education, 26, 2, (June 2001), 151-168. Publisher: Taylor & Francis, UK.
• 38: Maurizio Morisio , Ioannis Stamelos , Vasilis Spahos , Daniele Romano, Measuring Functionality and Productivity in Web-Based Applications: A Case Study, Proceedings of the 6th International Symposium on Software Metrics, p.111, November 04-06, 1999
• 39: Ingunn Myrtveit , Erik Stensrud, A Controlled Experiment to Assess the Benefits of Estimating with Analogy and Regression Models, IEEE Transactions on Software Engineering, v.25 n.4, p.510-525, July 1999 [doi>10.1109/32.799947]
• 40: Putnam, L. H., A General Empirical Solution to the Macro Sizing and Estimating Problem, IEEE Transactions on Software Engineering, SE-4, 4, 345 - 361, 1978.
• 41: Sylvie Ranwez , Torsten Leidig , Michel Crampes, Formalization to improve lifelong learning, Journal of Interactive Learning Research, v.11 n.3/4, p.389-409, Dec. 1, 2000
• 42: Donald J. Reifer, Web Development: Estimating Quick-to-Market Software, IEEE Software, v.17 n.6, p.57-64, November 2000 [doi>10.1109/52.895169]
• 43: Schofield, C. An empirical investigation into software estimation by analogy, PhD thesis, Dept. of Computing, Bournemouth University, UK, 1998.
• 44: Schulz, S. CBR-Works - A State-of-the-Art Shell for Case-Based Application Building, Proceedings of the German Workshop on Case-Based Reasoning, GWCBR'99 (1999).
• 45: Schwabe, D. and Rossi, G., From Domain Models to Hypermedia Applications: An Object-Oriented Approach, Proceedings of the International Workshop on Methodologies for Designing and Developing Hypermedia Applications, Edimburgh, (September 1994).
• 46: R. W. Selby , A. A. Porter, Learning from Examples: Generation and Evaluation of Decision Trees for Software Resource Analysis, IEEE Transactions on Software Engineering, v.14 n.12, p.1743-1757, December 1988 [doi>10.1109/32.9061]
• 47: Martin Shepperd , Gada Kadoda, Using Simulation to Evaluate Prediction Techniques, Proceedings of the 7th International Symposium on Software Metrics, p.349, April 04-06, 2001
• 48: Martin Shepperd , Chris Schofield, Estimating Software Project Effort Using Analogies, IEEE Transactions on Software Engineering, v.23 n.11, p.736-743, November 1997 [doi>10.1109/32.637387]
• 49: Martin Shepperd , Chris Schofield , Barbara Kitchenham, Effort estimation using analogy, Proceedings of the 18th international conference on Software engineering, p.170-178, March 25-29, 1996, Berlin, Germany
• 50: Spiro, R. J., Feltovich, P. J., Jacobson, M. J., and Coulson, R. L., Cognitive Flexibility, Constructivism, and Hypertext: Random Access Instruction for Advanced Knowledge Acquisition in Ill-Structured Domains, In: L. Steffe & J. Gale, eds., Constructivism, Hillsdale, N.J.:Erlbaum, 1995.
• 51: Krishnamoorthy Srinivasan , Douglas Fisher, Machine Learning Approaches to Estimating Software Development Effort, IEEE Transactions on Software Engineering, v.21 n.2, p.126-137, February 1995 [doi>10.1109/32.345828]
• 52: Susana Pajares Tosca, The lyrical quality of links, Proceedings of the tenth ACM Conference on Hypertext and hypermedia : returning to our diverse roots: returning to our diverse roots, p.217-218, February 21-25, 1999, Darmstadt, Germany [doi>10.1145/294469.294911]
• 53: Ian Watson, Applying case-based reasoning: techniques for enterprise systems, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1998

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.0 General

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.8 Metrics

General Terms:
Experimentation, Measurement

Keywords:
case-based reasoning, prediction models, web effort prediction, web hypermedia, web hypermedia metrics