A correlation-based model for unsupervised feature selection

ABSTRACT

We propose a new model for feature evaluation and selection that assesses the propensity of the features to support two-set classification. For each item of the data set, the collection of features induce a ranking (ordered list) of the remaining items. The evaluation criterion favors features that result in the most consistent discrimination between relevant and non-relevant items within these ranked lists. The discrimination boundaries within a single list are determined combinatorially, according to the degree of correlation among the relevant sets of its members. The model makes no special assumptions on the nature of the data. A selection heuristic based on the model is also proposed using sequential forward generation, and an experimental comparison is made with other unsupervised feature selection methods.

REFERENCES

• 1: N. Boujemaa, J. Fauqueur, M. Ferecatu, F. Fleuret, V. Gouet, B. Le Saux and H. Sahbi, IKONA: interactive generic and specific image retrieval, Proc. Intern. Workshop on Multimedia Content-based Indexing and Retrieval (MMCBIR), Rocquencourt, France, 2001.
• 2: Dimensionality Reduction of Unsupervised Data, Proceedings of the 9th International Conference on Tools with Artificial Intelligence, p.532, November 03-08, 1997
• 3: Mark Devaney , Ashwin Ram, Efficient Feature Selection in Conceptual Clustering, Proceedings of the Fourteenth International Conference on Machine Learning, p.92-97, July 08-12, 1997
• 4: Jennifer G. Dy , Carla E. Brodley, Feature Selection for Unsupervised Learning, The Journal of Machine Learning Research, 5, p.845-889, 12/1/2004
• 5: M. Ferecatu, Image Retrieval With Active Relevance Feedback Using Both Visual and Keyword-based Descriptors, Ph.D. thesis,University of Versailles Saint-Quentin-en-Yvelines, July 2005.
• 6: Keinosuke Fukunaga, Introduction to statistical pattern recognition (2nd ed.), Academic Press Professional, Inc., San Diego, CA, 1990
• 7: Jan-Mark Geusebroek , Gertjan J. Burghouts , Arnold W. M. Smeulders, The Amsterdam Library of Object Images, International Journal of Computer Vision, v.61 n.1, p.103-112, January 2005 [doi>10.1023/B:VISI.0000042993.50813.60]
• 8: M. E. Houle, Clustering without data: the relevant set correlation model, Proc. International Workshop on Data-Mining and Statistical Science(DMSS 2006), Sapporo, Japan, September 2006, pp. 54--61.
• 9: M. E. Houle, Clustering without data: the GreedyRSC heuristic, Proc. International Workshop on Data-Mining and Statistical Science(DMSS 2006), Sapporo, Japan, September 2006, pp. 62--69.
• 10: YeongSeog Kim , W. Nick Street , Filippo Menczer, Feature selection in unsupervised learning via evolutionary search, Proceedings of the sixth ACM SIGKDD international conference on Knowledge discovery and data mining, p.365-369, August 20-23, 2000, Boston, Massachusetts, United States [doi>10.1145/347090.347169]
• 11: Huan Liu , Hiroshi Motoda, Feature Selection for Knowledge Discovery and Data Mining, Kluwer Academic Publishers, Norwell, MA, 1998
• 12: Huan Liu , Lei Yu, Toward Integrating Feature Selection Algorithms for Classification and Clustering, IEEE Transactions on Knowledge and Data Engineering, v.17 n.4, p.491-502, April 2005 [doi>10.1109/TKDE.2005.66]
• 13: Pabitra Mitra , C. A. Murthy , Sankar K. Pal, Unsupervised Feature Selection Using Feature Similarity, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.3, p.301-312, March 2002 [doi>10.1109/34.990133]
• 14: S. A. Nene, S. K. Nayar and H. Murase, Columbia object image library, Technical Report, Department of Computer Science, Columbia University, 1996(http://www.cs.columbia.edu/CAVE/).
• 15: Ning Zhong , Juzhen Dong , Setsuo Ohsuga, Using Rough Sets with Heuristics for Feature Selection, Journal of Intelligent Information Systems, v.16 n.3, p.199-214, August 2001 [doi>10.1023/A:1011219601502]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

General Terms:
Performance

Keywords:
clustering, feature selection