Continuous visual vocabulary modelsfor pLSA-based scene recognition

ABSTRACT

Topic models such as probabilistic Latent Semantic Analysis (pLSA) and Latent Dirichlet Allocation (LDA) have been shown to perform well in various image content analysis tasks. However, due to the origin of these models from the text domain, almost all prior work uses discrete vocabularies even when applied in the image domain. Thus in these works the continuous local features used to describe an image need to be quantized to fit the model. In this work we will propose and evaluate three different extensions to the pLSA framework so that words are modeled as continuous feature vector distributions rather than crudely quantized high-dimensional descriptors. The performance of these continuous vocabulary models are compared in an automatic scene recognition task. Our experiments clearly show that the continuous approaches outperform the standard pLSA model. In this paper all required equations for parameter estimation and inference are given for each of the three models.

REFERENCES

• 1: P. Ahrendt, C. Goutte, and J. Larsen. Co-occurrence models in music genre classification. In IEEE International Workshop on Machine Learning for Signal Processing, pages 247--252, 2005.
• 2: Kobus Barnard , Pinar Duygulu , David Forsyth , Nando de Freitas , David M. Blei , Michael I. Jordan, Matching words and pictures, The Journal of Machine Learning Research, 3, 3/1/2003
• 3: David M. Blei , Michael I. Jordan, Modeling annotated data, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860460]
• 4: David M. Blei , Andrew Y. Ng , Michael I. Jordan, Latent dirichlet allocation, The Journal of Machine Learning Research, 3, p.993-1022, 3/1/2003 [doi>10.1162/jmlr.2003.3.4-5.993]
• 5: A. Bosch, A. Zisserman, and X. Munoz. Scene classification via pLSA. In Proceedings of the European Conference on Computer Vision, 2006.
• 6: L. Cao and L. Fei-Fei. Spatially coherent latent topic model for concurrent object segmentation and classification. In IEEE Intern. Conf. on Computer Vision (ICCV), 2007.
• 7: A. Dempster, N. Laird, and D. Rubin. Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society, 39(1):1--38, 1977.
• 8: Thomas Hofmann, Unsupervised learning by probabilistic latent semantic analysis, Machine Learning, v.42 n.1/2, p.177-196, Jan. 2001 [doi>10.1023/A:1007617005950]
• 9: Eva Hörster , Rainer Lienhart , Malcolm Slaney, Image retrieval on large-scale image databases, Proceedings of the 6th ACM international conference on Image and video retrieval, p.17-24, July 09-11, 2007, Amsterdam, The Netherlands [doi>10.1145/1282280.1282283]
• 10: D. Larlus and F. Jurie. Latent mixture vocabularies for object categorization. In British Machine Vision Conference, 2006.
• 11: Fei-Fei Li , Pietro Perona, A Bayesian Hierarchical Model for Learning Natural Scene Categories, Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2, p.524-531, June 20-26, 2005 [doi>10.1109/CVPR.2005.16]
• 12: R. Lienhart and M. Slaney. pLSA on large scale image databases. In IEEE International Conference on Acoustics, Speech and Signal Processing, 2007.
• 13: R. Lienhart and M. Slaney. pLSA on large scale image databases. In IEEE International Conference on Acoustics, Speech and Signal Processing, 2007.
• 14: Aude Oliva , Antonio Torralba, Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope, International Journal of Computer Vision, v.42 n.3, p.145-175, May-June 2001 [doi>10.1023/A:1011139631724]
• 15: P. Quelhas , F. Monay , J.-M. Odobez , D. Gatica-Perez , T. Tuytelaars , L. Van Gool, Modeling Scenes with Local Descriptors and Latent Aspects, Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1, p.883-890, October 17-20, 2005 [doi>10.1109/ICCV.2005.152]
• 16: Josef Sivic , Bryan C. Russell , Alexei A. Efros , Andrew Zisserman , William T. Freeman, Discovering Objects and their Localization in Images, Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1, p.370-377, October 17-20, 2005 [doi>10.1109/ICCV.2005.77]
• 17: Martin Szummer , Rosalind W. Picard, Indoor-Outdoor Image Classification, Proceedings of the 1998 International Workshop on Content-Based Access of Image and Video Databases (CAIVD '98), p.42, January 03-03, 1998
• 18: A. Vailaya, M. Figueiredo, A. Jain, and H. Zhang. Image classification for content-based indexing. IEEE Transactions on Image Processing, 10(1):117--130, 2001.
• 19: J. Vogel and B. Schiele. Natural scene retrieval based on a semantic modeling step. In CIVR, pages 207--215, 2004.
• 20: S. Young. A review of large-vocabulary continuous-speech recognition. IEEE Signal Processing Magazine, 13(5):45--57, 1996.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.4 Applications
          Subjects: Computer vision

Additional Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
        I.4.10 Image Representation
            Subjects: Statistical

General Terms:
Algorithms, Experimentation, Theory

Keywords:
continuous visual vocabulary, pLSA, scene recognition