Heidi matrix: nearest neighbor driven high dimensional data visualization

ABSTRACT

Identifying patterns in large high dimensional data sets is a challenge. As the number of dimensions increases, the patterns in the data sets tend to be more prominent in the subspaces than the original dimensional space. A system to facilitate presentation of such subspace oriented patterns in high dimensional data sets is required to understand the data.

Heidi is a high dimensional data visualization system that captures and visualizes the closeness of points across various subspaces of the dimensions; thus, helping to understand the data. The core concept behind Heidi is based on prominence of patterns within the nearest neighbor relations between pairs of points across the subspaces.

Given a d-dimensional data set as input, Heidi system generates a 2-D matrix represented as a color image. This representation gives insight into (i) how the clusters are placed with respect to each other, (ii) characteristics of placement of points within a cluster in all the subspaces and (iii) characteristics of overlapping clusters in various subspaces.

A sample of results displayed and discussed in this paper illustrate how Heidi Visualization can be interpreted.

REFERENCES

• 1: Charu C. Aggarwal , Alexander Hinneburg , Daniel A. Keim, On the Surprising Behavior of Distance Metrics in High Dimensional Spaces, Proceedings of the 8th International Conference on Database Theory, p.420-434, January 04-06, 2001
• 2: Charu C. Aggarwal , Joel L. Wolf , Philip S. Yu , Cecilia Procopiuc , Jong Soo Park, Fast algorithms for projected clustering, Proceedings of the 1999 ACM SIGMOD international conference on Management of data, p.61-72, May 31-June 03, 1999, Philadelphia, Pennsylvania, United States
• 3: Ira Assent , Ralph Krieger , Emmanuel Müller , Thomas Seidl, VISA: visual subspace clustering analysis, ACM SIGKDD Explorations Newsletter, v.9 n.2, December 2007 [doi>10.1145/1345448.1345451]
• 4: U. Axen and H. Edelsbrunner. Auditory morse analysis of triangulated manifolds. In Mathematical Visualization, pages 223--236. Springer-Verlag, 1998.
• 5: Christian Baumgartner , Claudia Plant , Karin Kailing , Hans-Peter Kriegel , Peer Kroger, Subspace Selection for Clustering High-Dimensional Data, Proceedings of the Fourth IEEE International Conference on Data Mining, p.11-18, November 01-04, 2004
• 6: C.-H. Chen. Generalized association plots for information visualization: The applications of the convergence of iteratively formed correlation matrices. volume 12, pages 1--23. Statistica Sinica, 2002.
• 7: M. Ester, H. P. Kriegel, J. Sander, and X. Xu. A density-based algorithm for discovering clusters in large spatial databases with noise. In Proc. ACM SIGMOD, pages 226--231, 1996.
• 8: K. Kailing, H. P. Kriegel, and P. Kroger. Density-connected subspace clustering for high-dimensional data. In Proc. ICDM, 2004.
• 9: Soujanya Vadapalli , Satyanarayana R. Valluri , Kamalakar Karlapalem, A Simple Yet Effective Data Clustering Algorithm, Proceedings of the Sixth International Conference on Data Mining, p.1108-1112, December 18-22, 2006 [doi>10.1109/ICDM.2006.9]
• 10: J. Vennam and S. Vadapalli. Syndeca: Synthetic generation of datasets to evaluate clustering algorithms. In COMAD, 2005.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.7 Three-Dimensional Graphics and Realism

Additional Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.6 Methodology and Techniques
          Subjects: Interaction techniques
  I.6 SIMULATION AND MODELING
      I.6.5 Model Development
          Subjects: Modeling methodologies

General Terms:
Algorithms, Design, Theory