ICA and ISA using Schweizer-Wolff measure of dependence

ABSTRACT

We propose a new algorithm for independent component and independent subspace analysis problems. This algorithm uses a contrast based on the Schweizer-Wolff measure of pairwise dependence (Schweizer & Wolff, 1981), a non-parametric measure computed on pairwise ranks of the variables. Our algorithm frequently outperforms state of the art ICA methods in the normal setting, is significantly more robust to outliers in the mixed signals, and performs well even in the presence of noise. Our method can also be used to solve independent subspace analysis (ISA) problems by grouping signals recovered by ICA methods. We provide an extensive empirical evaluation using simulated, sound, and image data.

REFERENCES

• 1: Amari, S., Cichocki, A., & Yang, H. (1996). A new learning algorithm for blind source separation. NIPS (pp. 757--763).
• 2: Francis R. Bach , Michael I. Jordan, Kernel independent component analysis, The Journal of Machine Learning Research, 3, p.1-48, 3/1/2003 [doi>10.1162/153244303768966085]
• 3: Francis R. Bach , Michael I. Jordan, Beyond independent components: trees and clusters, The Journal of Machine Learning Research, 4, 12/1/2003
• 4: Cardoso, J.-F. (1998). Multidimensional independent component analysis. Proc. ICASSP'98, Seattle, WA.
• 5: Jean-François Cardoso, High-order contrasts for independent component analysis, Neural Computation, v.11 n.1, p.157-192, Jan. 1999 [doi>10.1162/089976699300016863]
• 6: Pierre Comon, Independent component analysis, a new concept?, Signal Processing, v.36 n.3, p.287-314, April 1994 [doi>10.1016/0165-1684(94)90029-9]
• 7: Deheuvels, P. (1979). La fonction de dépendance empirique et ses propriétés, un test non paramétrique d'indépendance. Bulletin de l'Académie Royale de Belgique, Classe des Sciences, 274--292.
• 8: Hyvärinen, A. (1999). Fast and robust fixed-point algorithms for independent component analysis. IEEE Trans. on Neural Networks, 626--634.
• 9: Hyvärinen, A., Karhunen, J., & Oja, E. (2001). Independent component analysis. New York: John Wiley.
• 10: Hyvärinen, A., & Kööster, U. (2006). FastISA: A fast fixed-point algorithm for independent subspace analysis. Proc. of ESANN.
• 11: Erik G. Learned-Miller , John W. Fisher, III, ICA using spacings estimates of entropy, The Journal of Machine Learning Research, v.4 n.7-8, p.1271-1295, October 1 - November 15, 2004 [doi>10.1162/jmlr.2003.4.7-8.1271]
• 12: Roger B. Nelsen, An Introduction to Copulas (Springer Series in Statistics), Springer-Verlag New York, Inc., Secaucus, NJ, 2006
• 13: Barnabás Póczos , András Lõrincz, Independent subspace analysis using geodesic spanning trees, Proceedings of the 22nd international conference on Machine learning, p.673-680, August 07-11, 2005, Bonn, Germany [doi>10.1145/1102351.1102436]
• 14: Schweizer, B., & Wolff, E. F. (1981). On nonparametric measures of dependence for random variables. The Annals of Statistics, 9, 879--885.
• 15: Sklar, A. (1959). Fonctions de répartition à n dimensions et leures marges. Publications de l'Institut de Statistique de L'Université de Paris, 8, 229--231.
• 16: Zoltán Szabó , Barnabás Póczos , András Lőrincz, Undercomplete Blind Subspace Deconvolution, The Journal of Machine Learning Research, 8, p.1063-1095, 5/1/2007
• 17: Theis, F. J. (2005). Blind signal separation into groups of dependent signals using joint block diagonalization. Proc. of ISCAS. (pp. 5878--5881).
• 18: Theis, F. J. (2007). Towards a general independent subspace analysis. Proc. of NIPS 19 (pp. 1361--1368).

INDEX TERMS

Primary Classification:
  G. Mathematics of Computing
  G.3 PROBABILITY AND STATISTICS

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning
  I.6 SIMULATION AND MODELING
      I.6.4 Model Validation and Analysis
      I.6.5 Model Development
          Subjects: Modeling methodologies

General Terms:
Algorithms, Design, Performance, Theory