Mining software effort data: preliminary analysis of visual studio team system data

ABSTRACT

In the software development process, scheduling and predictability are important components to delivering a product on time and within budget. Effort estimation artifacts offer a rich data set for improving scheduling accuracy and for understanding the development process. Effort estimation data for 55 features in the latest release of Visual Studio Team System (VSTS) were collected and analyzed for trends, patterns, and differences. Statistical analysis shows that actual estimation error was positively correlated with feature size, and that in-process metrics of estimation error were also correlated with the final estimation error. These findings suggest that smaller features can be estimated more accurately, and that in-process estimation error metrics can be provide a quantitative supplement to developer intuition regarding high-risk features during the development process.

REFERENCES

• 1: A. J. Albrecht , J. E. Gaffney, Software Function, Source Lines of Code, and Development Effort Prediction: A Software Science Validation, IEEE Transactions on Software Engineering, v.9 n.6, p.639-648, November 1983 [doi>10.1109/TSE.1983.235271]
• 2: Barry W. Boehm, Software Engineering Economics, Prentice Hall PTR, Upper Saddle River, NJ, 1981
• 3: Barry W. Boehm , Clark , Horowitz , Brown , Reifer , Chulani , Ray Madachy , Bert Steece, Software Cost Estimation with Cocomo II with Cdrom, Prentice Hall PTR, Upper Saddle River, NJ, 2000
• 4: Lionel C. Briand , Khaled El Emam , Frank Bomarius, COBRA: a hybrid method for software cost estimation, benchmarking, and risk assessment, Proceedings of the 20th international conference on Software engineering, p.390-399, April 19-25, 1998, Kyoto, Japan
• 5: Magne Jorgensen, Practical Guidelines for Expert-Judgment-Based Software Effort Estimation, IEEE Software, v.22 n.3, p.57-63, May 2005 [doi>10.1109/MS.2005.73]
• 6: Magne Jorgensen , Kjetil Molokken-Ostvold, Reasons for Software Effort Estimation Error: Impact of Respondent Role, Information Collection Approach, and Data Analysis Method, IEEE Transactions on Software Engineering, v.30 n.12, p.993-1007, December 2004 [doi>10.1109/TSE.2004.103]
• 7: Magne Jrgensen , Dag I. K. Sjberg , Geir Kirkeben, The Prediction Ability of Experienced Software Maintainers, Proceedings of the Conference on Software Maintenance and Reengineering, p.93, February 29-March 03, 2000
• 8: Martin Shepperd , Chris Schofield, Estimating Software Project Effort Using Analogies, IEEE Transactions on Software Engineering, v.23 n.11, p.736-743, November 1997 [doi>10.1109/32.637387]

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.8 Metrics
          Subjects: Process metrics

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.8 Metrics
          Subjects: Performance measures; Product metrics

General Terms:
Experimentation, Measurement

Keywords:
effort estimation, prediction