Can data transformation help in the detection of fault-prone modules?

ABSTRACT

Data preprocessing (transformation) plays an important role in data mining and machine learning. In this study, we investigate the effect of four different preprocessing methods to fault-proneness prediction using nine datasets from NASA Metrics Data Programs (MDP) and ten classification algorithms. Our experiments indicate that log transformation rarely improves classification performance, but discretization affects the performance of many different algorithms. The impact of different transformations differs. Random forest algorithm, for example, performs better with original and log transformed data set. Boosting and NaiveBayes perform significantly better with discretized data. We conclude that no general benefit can be expected from data transformations. Instead, selected transformation techniques are recommended to boost the performance of specific classification algorithms.

REFERENCES

• 1: The R Project for Statistical Computing, available http://www.r-project.org/.
• 2: Metric data program. NASA Independent Verification and Validation facility, Available from http://MDP.ivv.nasa.gov.
• 3: Leo Breiman, Random Forests, Machine Learning, v.45 n.1, p.5-32, October 1 2001 [doi>10.1023/A:1010933404324]
• 4: W. J. Conover. Practical Nonparametric Statistics. John Wiley and Sons, Inc., 1999.
• 5: Janez Demšar, Statistical Comparisons of Classifiers over Multiple Data Sets, The Journal of Machine Learning Research, 7, p.1-30, 12/1/2006
• 6: J. Dougherty, R. Kohavi, and M. Sahami. Supervised and unsupervised discretization of continuous features. In International Conference on Machine Learning, pages 194--202, 1995.
• 7: J. J. Faraway. Practical Regression and Anova using R. online, July 2002.
• 8: U. M. Fayyad and K. B. Irani. Multi-interval discretization of continuous-valued attributes for classification learning, pages 1022--1027, 1993.
• 9: Isabelle Guyon , Steve Gunn , Masoud Nikravesh , Lotfi A. Zadeh, Feature Extraction: Foundations and Applications (Studies in Fuzziness and Soft Computing), Springer-Verlag New York, Inc., Secaucus, NJ, 2006
• 10: Yue Jiang , Bojan Cukic , Tim Menzies, Fault Prediction using Early Lifecycle Data, Proceedings of the The 18th IEEE International Symposium on Software Reliability, p.237-246, November 05-09, 2007 [doi>10.1109/ISSRE.2007.9]
• 11: I. Jolliffe. Principal Component Analysis. Springer, New York, 2002.
• 12: Tim Menzies , Jeremy Greenwald , Art Frank, Data Mining Static Code Attributes to Learn Defect Predictors, IEEE Transactions on Software Engineering, v.33 n.1, p.2-13, January 2007 [doi>10.1109/TSE.2007.10]
• 13: S. Siegel. Nonparametric Satistics. New York: McGraw-Hill Book Company, Inc., 1956.
• 14: Ian H. Witten , Eibe Frank, Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems), Morgan Kaufmann Publishers Inc., San Francisco, CA, 2005

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications
          Subjects: Data mining

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning
  I.5 PATTERN RECOGNITION
      I.5.2 Design Methodology
          Subjects: Classifier design and evaluation

General Terms:
Algorithms, Design, Experimentation, Measurement, Performance, Reliability