GAODE and HAODE: two proposals based on AODE to deal with continuous variables

ABSTRACT

AODE (Aggregating One-Dependence Estimators) is considered one of the most interesting representatives of the Bayesian classifiers, taking into account not only the low error rate it provides but also its efficiency. Until now, all the attributes in a dataset have had to be nominal to build an AODE classifier or they have had to be previously discretized. In this paper, we propose two different approaches in order to deal directly with numeric attributes. One of them uses conditional Gaussian networks to model a dataset exclusively with numeric attributes; and the other one keeps the superparent on each model discrete and uses univariate Gaussians to estimate the probabilities for the numeric attributes and multinomial distributions for the categorical ones, it also being able to model hybrid datasets. Both of them obtain competitive results compared to AODE, the latter in particular being a very attractive alternative to AODE in numeric datasets.

REFERENCES

• 1: Ethem Alpaydin, Combined 5 × 2 cv F test for comparing supervised classification learning algorithms, Neural Computation, v.11 n.9, p.1885-1892, Nov. 15, 1999 [doi>10.1162/089976699300016007]
• 2: Andersen, S. K., Olesen, K. G., Jensen, F. V., & Jensen, F. (1989). HUGIN-A shell for building Bayesian belief universes for expert systems. Proc. of the 11th Int. Joint Conf. on AI (pp. 1080--1085).
• 3: Asuncion, A., & Newman, D. (2007). UCI machine learning repository. University of California, Irvine, School of Information and Computer Sciences. http://www.ics.uci.edu/~mlearn/MLRepository.html.
• 4: DeGroot, M. H. (1970). Optimal statistical decisions. New York: McGraw-Hill.
• 5: Janez Demšar, Statistical Comparisons of Classifiers over Multiple Data Sets, The Journal of Machine Learning Research, 7, p.1-30, 12/1/2006
• 6: Domingos, P., & Pazzani, M. (1996). Beyond independence: Conditions for the optimality of the simple Bayesian classifier. Proc. of the 13th Int. Conf. on Machine Learning (pp. 105--112).
• 7: Duda, R. O., Hart, P. E., & Stork, D. G. (1973). Pattern classification and scene analysis. Wiley NY.
• 8: Fayyad, U. M., & Irani, K. B. (1993). Multi-interval discretization of continuous-valued attributes for classification learning. Proc. of the 13th Int. Joint Conf. on Articial Intelligence (pp. 1022--1027).
• 9: García, S., & Herrera, F. (2009). An extension on "statistical comparisons of classifiers over multiple data sets" for all pairwise comparisons. J. Mach. Learn. Res., 9, 2677--2694.
• 10: Geiger, D., & Heckerman, D. (1994). Learning Gaussian networks. Proc. of the 10th Annual Conf. on Uncertainty in AI (pp. 235--243).
• 11: Keogh, E., & Pazzani, M. (1999). Learning augmented Bayesian classifiers: A comparison of distribution-based and classification-based approaches. Proc. of the 7th Int. Workshop on AI and Statistics (pp. 225--230).
• 12: Langley, P., Iba, W., & Thompson, K. (1992). An analysis of Bayesian classifiers. Proc. of the 10th National Conf. on AI (pp. 223--228).
• 13: Larrañaga, P., Etxeberria, R., Lozano, J., & Peñña, J. M. (1999). Optimization by learning and simulation of Bayesian and Gaussian networks (Technical Report). University of the Basque Country.
• 14: Steffen L. Lauritzen , Frank Jensen, Stable local computation with conditional Gaussian distributions, Statistics and Computing, v.11 n.2, p.191-203, April 2001 [doi>10.1023/A:1008935617754]
• 15: Serafín Moral , Rafael Rumí , Antonio Salmerón, Mixtures of Truncated Exponentials in Hybrid Bayesian Networks, Proceedings of the 6th European Conference on Symbolic and Quantitative Approaches to Reasoning with Uncertainty, p.156-167, September 19-21, 2001
• 16: Richard E. Neapolitan, Learning Bayesian Networks, Prentice-Hall, Inc., Upper Saddle River, NJ, 2003
• 17: Pérez, A., Larrañaga, P., & Inza, I. (2006). Supervised classification with conditional gaussian networks: Increasing the structure complexity from naive Bayes. Int. J. Approx. Reasoning, 43, 1--25.
• 18: Sahami, M. (1996). Learning limited dependence Bayesian classifiers. Proc. of the 2nd Int. Conf. on Knowledge Discovery in Databases (pp. 335--338).
• 19: Geoffrey I. Webb , Janice R. Boughton , Zhihai Wang, Not So Naive Bayes: Aggregating One-Dependence Estimators, Machine Learning, v.58 n.1, p.5-24, January 2005 [doi>10.1007/s10994-005-4258-6]
• 20: Wek08 (2008). Collection of datasets avalaibles from the weka official homepage. http://www.cs.waikato.ac.nz/ml/weka/.
• 21: Ian H. Witten , Eibe Frank, Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems), Morgan Kaufmann Publishers Inc., San Francisco, CA, 2005
• 22: Xindong Wu , Vipin Kumar , J. Ross Quinlan , Joydeep Ghosh , Qiang Yang , Hiroshi Motoda , Geoffrey J. McLachlan , Angus Ng , Bing Liu , Philip S. Yu , Zhi-Hua Zhou , Michael Steinbach , David J. Hand , Dan Steinberg, Top 10 algorithms in data mining, Knowledge and Information Systems, v.14 n.1, p.1-37, December 2007 [doi>10.1007/s10115-007-0114-2]
• 23: Zheng, F., & Webb, G. (2005). A Comparative Study of Semi-naive Bayes Methods in Classification Learning. Proc. of the 4th Australian Data Mining Conf. (pp. 141--156).
• 24: Zijian Zheng , Geoffrey I. Webb, Lazy Learning of Bayesian Rules, Machine Learning, v.41 n.1, p.53-84, Oct. 2000 [doi>10.1023/A:1007613203719]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.2 Design Methodology
          Subjects: Classifier design and evaluation

Additional Classification:
  G. Mathematics of Computing
  G.3 PROBABILITY AND STATISTICS
      Subjects: Probabilistic algorithms (including Monte Carlo)

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning
  I.6 SIMULATION AND MODELING
      I.6.4 Model Validation and Analysis
      I.6.5 Model Development
          Subjects: Modeling methodologies

General Terms:
Algorithms, Design, Performance, Theory