Genetic fuzzy discretization with adaptive intervals for classification problems

ABSTRACT

We propose a genetic fuzzy discretization method for continuous numerical attributes. Traditional discretization methods categorize the continuous attributes into a number of bins. Because they are made on crisp discretization, there exists considerable information loss. Fuzzy discretization allows overlapping intervals and reflects linguistic classification. However, the number of intervals, the boundaries of intervals, and the degrees of overlapping are intractable to get optimized. We use a genetic algorithm to optimize these parameters. Experimental results showed considerable improvement on the classification accuracy over a crisp discretization and a typical fuzzy discretization.

REFERENCES

• 1: Petr Berka , Ivan Bruha, Discretization and Grouping: Preprocessing Steps for Data Mining, Proceedings of the Second European Symposium on Principles of Data Mining and Knowledge Discovery, p.239-245, September 23-26, 1998
• 2: Thang Nguyen Bui , Byung Ro Moon, Genetic Algorithm and Graph Partitioning, IEEE Transactions on Computers, v.45 n.7, p.841-855, July 1996 [doi>10.1109/12.508322]
• 3: Jason Catlett, On Changing Continuous Attributes into Ordered Discrete Attributes, Proceedings of the European Working Session on Machine Learning, p.164-178, March 06-08, 1991
• 4: S. M. Chen. Interval-valued fuzzy hypergraph and fuzzy partition. IEEE Trans. on Systems, Man and Cybernetics, Part B 27(4):725--733, 1997.
• 5: John Y. Ching , Andrew K. C. Wong , Keith C. C. Chan, Class-Dependent Discretization for Inductive Learning from Continuous and Mixed-Mode Data, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.17 n.7, p.641-651, July 1995 [doi>10.1109/34.391407]
• 6: David K. Y. Chiu, Andrew K. C. Wong, and B. Cheung. In formation discovery through hierarchical maximum entropy discretization and synthesis. In Knowledge Discovery in Databases pages 125--140. 1991.
• 7: B. Efron and R. Tibshirani. Cross-validation and the bootstrap: Estimating the error rate of a prediction rule. In Technical Report(TR-477), Dept. of Statistics, Stanford University., 1995.
• 8: Tapio Elomaa , Juho Rousu, General and Efficient Multisplitting of Numerical Attributes, Machine Learning, v.36 n.3, p.201-244, Sept. 1999 [doi>10.1023/A:1007674919412]
• 9: U. M. Fayyad and K. B. Irani. Multi-interval discretization of continuous attributes as preprocessing or classification learning. In Proceedings of the Thirteenth International Joint Conference on Artificial Intelligence, Morgan Kaufmann pages 1022--1027, 1995.
• 10: D. E. Goldberg, K. Deb, and B. Korb. Do not worry, be messy. In Proceedings of the Fourth International Conference on Genetic Algorithms pages 24--30, 1991.
• 11: Hisao Ishibuchi , Takashi Yamamoto, Fuzzy Rule Selection By Data Mining Criteria And Genetic Algorithms, Proceedings of the Genetic and Evolutionary Computation Conference, p.399-406, July 09-13, 2002
• 12: H. Ishibuchi and T. Yamamoto. Deriving fuzzy discretization from interval discretization. In Proc. of 2003 IEEE International Conference on Fuzzy Systems pages 749--754, 2003.
• 13: Hisao Ishibuchi , Takashi Yamamoto , Tomoharu Nakashima, Fuzzy Data Mining: Effect of Fuzzy Discretization, Proceedings of the 2001 IEEE International Conference on Data Mining, p.241-248, November 29-December 02, 2001
• 14: R. Kohavi. A study of cross-validation and bootstrap of accuracy estimation and model selection. In Proceedings of hte Rourteenth International Joint Conference on Artificial Intelligence pages 1137--1143, 1995.
• 15: A. Kusiak. Feature transformation methods in data mining. IEEE Trans. on Electronics Packaging Manufacturing 24(3):214--221, 2001.
• 16: Huan Liu , Farhad Hussain , Chew Lim Tan , Manoranjan Dash, Discretization: An Enabling Technique, Data Mining and Knowledge Discovery, v.6 n.4, p.393-423, October 2002 [doi>10.1023/A:1016304305535]
• 17: T. Murata, H. Ishibuchi, and M. Gen. Adjusting fuzzy partitions by genetic algorithms and histograms or pattern classification problems. In Proc. of 1998 IEEE International Conference on Evolutionary Computation pages 9--14, 1998.
• 18: J. Ross Quinlan, C4.5: programs for machine learning, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1993
• 19: J. R. Quinlan. Improved use of continuous atrributes in c4. 5. Journal of Artificial Intelligence Research 4:77--90, 1996.
• 20: D. W. Stashuk and R. K. Naphan. Probabilistic inference-based classification applied to myoelectric signal decomposition. IEEE Transactions on Biomedical Engineering 39(4):346--355, 1992.
• 21: Gilbert Syswerda, Uniform Crossover in Genetic Algorithms, Proceedings of the 3rd International Conference on Genetic Algorithms, p.2-9, June 01, 1989
• 22: S. Trautzsch and P. Perner. A comparision of different multi-interval discretization methods or decision tree learning.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.1 Models
          Subjects: Fuzzy set

General Terms:
Experimentation, Performance

Keywords:
classification problem, fuzzy discretization, genetic algorithm