Visualizing changes in the structure of data for exploratory feature selection

ABSTRACT

Using visualization techniques to explore and understand high-dimensional data is an efficient way to combine human intelligence with the immense brute force computation power available nowadays. Several visualization techniques have been developed to study the cluster structure of data, i.e., the existence of distinctive groups in the data and how these clusters are related to each other. However, only few of these techniques lend themselves to studying how this structure changes if the features describing the data are changed. Understanding this relationship between the features and the cluster structure means understanding the features themselves and is thus a useful tool in the feature extraction phase.In this paper we present a novel approach to visualizing how modification of the features with respect to weighting or normalization changes the cluster structure. We demonstrate the application of our approach in two music related data mining projects.

REFERENCES

• 1: Charu C. Aggarwal, A human-computer cooperative system for effective high dimensional clustering, Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining, p.221-226, August 26-29, 2001, San Francisco, California [doi>10.1145/502512.502542]
• 2: An Intuitive Framework for Understanding Changes in Evolving Data Streams, Proceedings of the 18th International Conference on Data Engineering, p.261, February 26-March 01, 2002
• 3: Charu C. Aggarwal, Towards effective and interpretable data mining by visual interaction, ACM SIGKDD Explorations Newsletter, v.3 n.2, January 2002 [doi>10.1145/507515.507518]
• 4: Charu C. Aggarwal , Alexander Hinneburg , Daniel A. Keim, On the Surprising Behavior of Distance Metrics in High Dimensional Spaces, Proceedings of the 8th International Conference on Database Theory, p.420-434, January 04-06, 2001
• 5: J.-J. Aucouturier and F. Pachet. Music Similarity Measures: What's the Use? In Proceedings of the 3rd International Conference on Music Information Retrieval (ISMIR'02), Paris, France, 2002.
• 6: Christopher M. Bishop , Markus Svensén , Christopher K. I. Williams, GTM: the generative topographic mapping, Neural Computation, v.10 n.1, p.215-234, Jan. 1, 1998 [doi>10.1162/089976698300017953]
• 7: P. Cano, M. Kaltenbrunner, F. Gouyon, and E. Batlle. On the Use of Fastmap for Audio Retrieval and Browsing. In Proceedings of the 3rd International Conference on Music Information Retrieval (ISMIR'02), Paris, France, 2002.
• 8: Guido J. Deboeck , Tuevo K. Kohonen , T. K. Kohonen, Visual Explorations in Finance, Springer-Verlag New York, Inc., Secaucus, NJ, 1998
• 9: Usama Fayyad , Gregory Piatetsky-Shapiro , Padhraic Smyth, The KDD process for extracting useful knowledge from volumes of data, Communications of the ACM, v.39 n.11, p.27-34, Nov. 1996 [doi>10.1145/240455.240464]
• 10: Venkatesh Ganti , Johannes Gehrke , Raghu Ramakrishnan, A framework for measuring changes in data characteristics, Proceedings of the eighteenth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, p.126-137, May 31-June 03, 1999, Philadelphia, Pennsylvania, United States [doi>10.1145/303976.303989]
• 11: H. Hotelling. Analysis of a Complex of Statistical Variables into Principal Components. Journal of Educational Psychology, 24:417--441 and 498--520, 1933.
• 12: Eser Kandogan, Visualizing multi-dimensional clusters, trends, and outliers using star coordinates, Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining, p.107-116, August 26-29, 2001, San Francisco, California [doi>10.1145/502512.502530]
• 13: S. Kaski. Data Exploration Using Self-Organizing Maps. PhD thesis, Helsinki University of Technology, Department of Computer Science and Engineering, 1997.
• 14: S. Kaski. Fast Winner Search for SOM-Based Monitoring and Retrieval of High-Dimensional Data. In Proceedings of the 9th International Conference on Artificial Neural Networks (ICANN'99), London, UK, 1999.
• 15: S. Kaski, T. Honkela, K. Lagus, and T. Kohonen. WEBSOM -- Self-Organizing Maps of Document Collections. Neurocomputing, 21:101--117, 1998.
• 16: Metrics that Learn Relevance, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks (IJCNN'00)-Volume 5, p.5547, July 24-27, 2000
• 17: T. Kohonen. Self-Organizing Formation of Topologically Correct Feature Maps. Biological Cybernetics, 43:59--69, 1982.
• 18: T. Kohonen. The Adaptive-Subspace SOM (ASSOM) and its use for the Implementation of Invariant Feature Detection. In Proceedings of the International Conference on Artificial Neural Networks (ICANN'95), Paris, France, 1995.
• 19: Teuvo Kohonen, Self-organizing maps, Springer-Verlag New York, Inc., Secaucus, NJ, 1997
• 20: T. Kohonen, S. Kaski, K. Lagus, J. Salojärvi, J. Honkela, V. Paatero, and A. Saarela. Self Organization of a Massive Document Collection. IEEE Transactions on Neural Networks, Special Issue on Neural Networks for Data Mining and Knowledge Discovery, 11(3):574--585, 2000.
• 21: J. B. Kruskal and M. Wish. Multidimensional Scaling. Number 07--011 in Paper Series on Quantitative Applications in the Social Sciences. Sage Publications, Newbury Park, CA, 1978.
• 22: K. Lagus and S. Kaski. Keyword Selection Method for Characterizing Text Document Maps. In Proceedings of the 9th International Conference on Artificial Neural Networks (ICANN'99), London, UK, 1999.
• 23: Jörg Langner , Werner Goebl, Visualizing Expressive Performance in Tempo-Loudness Space, Computer Music Journal, v.27 n.4, p.69-83, December 2003 [doi>10.1162/014892603322730514]
• 24: B. Logan. Mel Frequency Cepstral Coefficients for Music Modeling. In Proceedings of the International Symposium on Music Information Retrieval (ISMIR), Plymouth, MA, 2000.
• 25: J. MacQueen. Some Methods for Classification and Analysis of Multivariate Observations. In Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability, Berkeley and Los Angeles, CA, 1967.
• 26: E. Pampalk. Islands of Music: Analysis, Organization, and Visualization of Music Archives. Master's thesis, Vienna University of Technology, 2001. http://www.oefai.at/~elias/music/thesis.html.
• 27: Elias Pampalk , Andreas Rauber , Dieter Merkl, Content-based organization and visualization of music archives, Proceedings of the tenth ACM international conference on Multimedia, December 01-06, 2002, Juan-les-Pins, France [doi>10.1145/641007.641121]
• 28: Elias Pampalk , Andreas Rauber , Dieter Merkl, Using Smoothed Data Histograms for Cluster Visualization in Self-Organizing Maps, Proceedings of the International Conference on Artificial Neural Networks, p.871-876, August 28-30, 2002
• 29: A. Rauber. LabelSOM: On the Labeling of Self-Organizing Maps. In Proceedings of the International Joint Conference on Neural Networks (IJCNN'99), Washington, DC, 1999.
• 30: J. W. Sammon. A Nonlinear Mapping for Data Structure Analysis. IEEE Transactions on Computers, 18:401--409, 1969.
• 31: A. Ultsch and H. P. Siemon. Kohonen's Self-Organizing Feature Maps for Exploratory Data Analysis. In Proceedings of the International Neural Network Conference (INNC'90), Dordrecht, Netherlands, 1990.
• 32: J. Vesanto and J. Ahola. Hunting for Correlations in Data Using the Self-Organizing Map. In Proceedings of the International ICSC Congress on Computational Intelligence Methods and Applications (CIMA'99), Rochester, NY, 1999.
• 33: J. Vesanto and E. Alhoniemi. Clustering of the Self-Organizing Map. IEEE Transactions on Neural Networks, 11(3):586--600, 2000.
• 34: Jörg A. Walter , Helge Ritter, On interactive visualization of high-dimensional data using the hyperbolic plane, Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, July 23-26, 2002, Edmonton, Alberta, Canada [doi>10.1145/775047.775065]
• 35: Gerhard Widmer, Using AI and machine learning to study expressive music performance: project survey and first report, AI Communications, v.14 n.3, p.149-162, September 2001
• 36: Gerhard Widmer, In Search of the Horowitz Factor: Interim Report on a Musical Discovery Project, Proceedings of the 5th International Conference on Discovery Science, p.13-21, November 24-26, 2002
• 37: E. Zwicker and H. Fastl. Psychoacoustics, Facts and Models. Springer, Berlin, Germany 2nd edition, 1999.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.3 Clustering
          Subjects: Similarity measures

Additional Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.3 Clustering
          Subjects: Algorithms

Keywords:
high-dimensional data, interactive data mining