Take a look at the new version of this page: [ beta version ]. Tell us what you think.

ABSTRACT

Five years ago the ISBSG database was used by Jeffery et al. [6] (S1) to compare the effort prediction accuracy between cross- and single-company effort models. Given that more than 2,000 projects were later volunteered to this database, in 2005 Mendes et al. [17] (S2) replicated S1 but obtained different results. The difference in results between both studies could have resulted from legitimate differences in data set patterns but also could have been influenced by differences in experimental procedure. S2 was unable to employ exactly the same experimental procedure used in S1, as S1's procedure was not fully documented. Therefore this paper aimed to apply S2's experimental procedure to the ISBSG database version used in S1 (release 6) to assess if differences in experimental procedure would have contributed towards different results. Our results corroborated those from S1: we found that predictions based on a single-company model were significantly more accurate than those based on a cross-company model.

REFERENCES

• 1: Lionel C. Briand , Khaled El Emam , Dagmar Surmann , Isabella Wieczorek , Katrina D. Maxwell, An assessment and comparison of common software cost estimation modeling techniques, Proceedings of the 21st international conference on Software engineering, p.313-322, May 16-22, 1999, Los Angeles, California, United States [doi>10.1145/302405.302647]
• 2: Lionel C. Briand , Tristen Langley , Isabella Wieczorek, A replicated assessment and comparison of common software cost modeling techniques, Proceedings of the 22nd international conference on Software engineering, p.377-386, June 04-11, 2000, Limerick, Ireland [doi>10.1145/337180.337223]
• 3: S. D. Conte , H. E. Dunsmore , Y. E. Shen, Software engineering metrics and models, Benjamin-Cummings Publishing Co., Inc., Redwood City, CA, 1986
• 4: Cook, R.D. Detection of influential observations in linear regression. Technometrics, 19, 1977, pp 15--18.
• 5: Jeffery, R., .M. Ruhe and I. Wieczorek. A Comparative Study of Two Software Development Cost Modeling Techniques using Multi-organizational and Companyspecific Data. Information and Software Technology, 42, 2000, pp 1009--1016.
• 6: Ross Jeffery , Melanie Ruhe , Isabella Wieczorek, Using Public Domain Metrics To Estimate Software Development Effort, Proceedings of the 7th International Symposium on Software Metrics, p.16, April 04-06, 2001
• 7: Chris F Kemerer, An empirical validation of software cost estimation models, Communications of the ACM, v.30 n.5, p.416-429, May 1987 [doi>10.1145/22899.22906]
• 8: Kitchenham, B.A., and E. Mendes. A Comparison of Crosscompany and Single-company Effort Estimation Models for Web Applications, Proceedings EASE 2004, 2004, pp 47--55.
• 9: Kitchenham, B.A., and E. Mendes, and Travassos, G. A Systematic Review of Cross- vs. Within-Company Cost Estimation Studies, EASE'06, (accepted for publication).
• 10: Kitchenham, B.A. and N.R. Taylor. Software cost models. ICL Technical Journal, May 1984, pp73--102.
• 11: Kirsopp, C. and Shepperd, M. Making Inferences with Small Numbers of Training Sets, IEE Proceedings Software, 149, pp 123--130, 2002.
• 12: Lefley, M., and M.J. Shepperd, Using Genetic Programming to Improve Software Effort Estimation Based on General Data Sets, Proceedings of GECCO 2003, LNCS 2724, Springer-Verlag, pp 2477--2487, 2003.
• 13: Lokan, C.J. Function Points. Advances in Computers, M.V. Zelkowitz (ed), Vol 65, pp 298--347, Elsevier, 2005.
• 14: Maxwell, K. Applied Statistics for Software Managers. Software Quality Institute Series, Prentice Hall, 2002.
• 15: Katrina Maxwell , Luk Van Wassenhove , Soumitra Dutta, Performance Evaluation of General and Company Specific Models in Software Development Effort Estimation, Management Science, v.45 n.6, p.787-803, June 1999 [doi>10.1287/mnsc.45.6.787]
• 16: Emilia Mendes , Barbara Kitchenham, Further Comparison of Cross-Company and Within-Company Effort Estimation Models for Web Applications, Proceedings of the Software Metrics, 10th International Symposium on (METRICS'04), p.348-357, September 11-17, 2004 [doi>10.1109/METRICS.2004.24]
• 17: Emilia Mendes , Chris Lokan , Robert Harrison , Chris Triggs, A Replicated Comparison of Cross-Company and Within-Company Effort Estimation Models Using the ISBSG Database, Proceedings of the 11th IEEE International Software Metrics Symposium (METRICS'05), p.36, September 19-22, 2005 [doi>10.1109/METRICS.2005.4]
• 18: Barbara G. Tabachnick , Linda S. Fidell, Using Multivariate Statistics (5th Edition), Allyn & Bacon, Inc., Needham Heights, MA, 2006
• 19: Isabella Wieczorek , Melanie Ruhe, How Valuable is company-specific Data Compared to multi-company Data for Software Cost Estimation?, Proceedings of the 8th International Symposium on Software Metrics, p.237, June 04-07, 2002

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.8 Metrics
          Subjects: Process metrics

General Terms:
Experimentation, Management, Measurement

Keywords:
cross-company estimation models, effort estimation, estimation models, experimental procedure, regression-based, replication study, single-company estimation model, software projects