Bayesian learning of measurement and structural models

ABSTRACT

We present a Bayesian search algorithm for learning the structure of latent variable models of continuous variables. We stress the importance of applying search operators designed especially for the parametric family used in our models. This is performed by searching for subsets of the observed variables whose covariance matrix can be represented as a sum of a matrix of low rank and a diagonal matrix of residuals. The resulting search procedure is relatively efficient, since the main search operator has a branch factor that grows linearly with the number of variables. The resulting models are often simpler and give a better fit than models based on generalizations of factor analysis or those derived from standard hill-climbing methods.

REFERENCES

• 1: Blake, C. L., & Merz, C. J. (1998). UCI repository, http://www.ics.uci.edu/~mlearn/mlrepository.html.
• 2: David M. Blei , Michael I. Jordan, Variational methods for the Dirichlet process, Proceedings of the twenty-first international conference on Machine learning, p.12, July 04-08, 2004, Banff, Alberta, Canada [doi>10.1145/1015330.1015439]
• 3: Bollen, K. (1989). Structural Equation Models with Latent Variables. John Wiley & Sons.
• 4: Drton, M., Sturmfels, B., & Sullivant, S. (2005). Algebraic factor analysis: tetrads, pentads and beyond. arxiv.org.
• 5: Elidan, G., Lotner, N., Friedman, N., & Koller, D. (2000). Discovering hidden variables. NIPS, 13.
• 6: Ghahramani, Z., & Beal, M. (1999). Variational inference for Bayesian mixture of factor analysers. NIPS, 12.
• 7: Kano, Y., & Harada, A. (2000). Stepwise variable selection in factor analysis. Psychometrika, 65.
• 8: Minka, T. (2000). Automatic choice of dimensionality for PCA. NIPS, 13.
• 9: Reyment, R., & Joreskog, K. (1996). Applied Factor Analysis in the Natural Sciences. Cambride Press.
• 10: Ricardo Silva , Richard Scheines, Automatic discovery of latent variable models, Carnegie Mellon University, Pittsburgh, PA, 2005
• 11: Ricardo Silva , Richard Scheines, New d-separation identification results for learning continuous latent variable models, Proceedings of the 22nd international conference on Machine learning, p.808-815, August 07-11, 2005, Bonn, Germany [doi>10.1145/1102351.1102453]
• 12: Silva, R., Scheines, R., Glymour, C., & Spirtes, P. (2006). Learning the structure of linear latent variable models. JMLR, 7, 191--246.
• 13: Spirtes, P., Glymour, C., & Scheines, R. (2000). Causation, Prediction and Search. Cambridge Press.
• 14: Jacob A. Wegelin , Asa Packer , Thomas S. Richardson, Latent models for cross-covariance, Journal of Multivariate Analysis, v.97 n.1, p.79-102, January 2006 [doi>10.1016/j.jmva.2004.11.009]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.1 Models
          Subjects: Structural

Additional Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.1 Models
          Subjects: Statistical
      I.5.2 Design Methodology

General Terms:
Algorithms