Hierarchical kernel stick-breaking process for multi-task image analysis

ABSTRACT

The kernel stick-breaking process (KSBP) is employed to segment general imagery, imposing the condition that patches (small blocks of pixels) that are spatially proximate are more likely to be associated with the same cluster (segment). The number of clusters is not set a priori and is inferred from the hierarchical Bayesian model. Further, KSBP is integrated with a shared Dirichlet process prior to simultaneously model multiple images, inferring their inter-relationships. This latter application may be useful for sorting and learning relationships between multiple images. The Bayesian inference algorithm is based on a hybrid of variational Bayesian analysis and local sampling. In addition to providing details on the model and associated inference framework, example results are presented for several image-analysis problems.

REFERENCES

• 1: Beal, M. (2003). Variational algorithms for approximate Bayesian inference. Doctoral dissertation, Gatsby Computational Neuroscience Unit, University College London.
• 2: David M. Blei , Michael I. Jordan, Variational methods for the Dirichlet process, Proceedings of the twenty-first international conference on Machine learning, p.12, July 04-08, 2004, Banff, Alberta, Canada [doi>10.1145/1015330.1015439]
• 3: Blei, D., & Lafferty, J. (2005). Correlated topic models. Advances in Neural Information Processing System.
• 4: Chris Ding , Xiaofeng He, K-means clustering via principal component analysis, Proceedings of the twenty-first international conference on Machine learning, p.29, July 04-08, 2004, Banff, Alberta, Canada [doi>10.1145/1015330.1015408]
• 5: Dunson, D., & Park, J.-H. (2008). Kernel stick-breaking process. Biometrika.
• 6: Ferguson, T. (1973). A bayesian analysis of some nonparametric problems. Annals of Statistics, 1.
• 7: Figueiredo, M., Cheng, D., & Murino, V. (2007). Clustering under prior knowledge with application to image segmentation. Advances in Neural Information Processing System.
• 8: Hoyer, P., & Hyväärinen, A. (2000). Independent component analysis applied to feature extraction from colour and stereo images. Network: Computation in Neural Systems, 11, 191--210.
• 9: Aapo Hyvärinen , Patrik Hoyer, Emergence of Phase- and Shift-Invariant Features by Decomposition of Natural Images into Independent Feature Subspaces, Neural Computation, v.12 n.7, p.1705-1720, July 2000 [doi>10.1162/089976600300015312]
• 10: Ishwaran, H., & James, L. (2001). Gibbs sampling methods for stick-breaking priors. Journal of the American Statistical Association, 96, 161--173.
• 11: MacEachern, S. (1999). Dependent nonparametric process. ASA Proceeding of the Section on Bayesian Statistical Science. Alexandria, VA.
• 12: McLachlan, G., & Basford, K. (1988). Mixture models: Inference and applications to clustering. Marcel Dekker.
• 13: Ng, A., Jordan, M., & Weiss, Y. (2001). On spectral clustering: Analysis and an algorithm. Advances in Neural Information Processing Systems 13.
• 14: Quelhas, P., F. Monay, J.-M. O., Gatica-Perez, D., & Tuytelaars, T. (2007). A thosand words in a scenes. IEEE Trans. Pattern Analysis Machine Intell., 9, 1575--1589.
• 15: Rasmussen, C. (2000). The infinite gaussian mixture model. Advances in Neural Information Processing System (pp. 554--560).
• 16: Sethuraman, J. (1994). A constructive definition of dirichlet priors. Statistica Sinica, 4, 639--650.
• 17: Sudderth, E. B., Torralba, A., Freeman, W. T., & Willsky, A. S. (2006). Describing visual scenes using transformed Dirichlet processes. NIPS 18 (pp. 1297--1304).
• 18: Teh, Y., Jordan, M., Beal, M., & Blei, D. (2005). Hierarchical dirichlet processes. Journal of the American Statistical Association, 101, 1566--1582.
• 19: Thrun, S., & O'Sullivan, J. (1996). Discovering structure in multiple learning tasks: The TC algorithm. Proc. the 13th International Conference on Machine Learning.
• 20: Wei, G., & Tanner, M. (1990). A monte carlo implementation of the em algorithm and the poor man's data augmentation algorithms. Journal of the American Statistical Association, 85, 699--704.
• 21: Ya Xue , Xuejun Liao , Lawrence Carin , Balaji Krishnapuram, Multi-Task Learning for Classification with Dirichlet Process Priors, The Journal of Machine Learning Research, 8, p.35-63, 5/1/2007

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.3 Deduction and Theorem Proving
          Subjects: Uncertainty, "fuzzy," and probabilistic reasoning; Inference engines
      I.2.6 Learning
  I.6 SIMULATION AND MODELING
      I.6.4 Model Validation and Analysis
      I.6.5 Model Development
          Subjects: Modeling methodologies

General Terms:
Algorithms, Design, Performance, Theory