Robust scene recognition using language models for scene contexts

ABSTRACT

We propose a robust scene recognition framework using scene context information for multimedia contents. Multimedia contents con-sist of scene sequences that are more likely to happen compared with other scene sequences. We employ a statistical approach to deal with this scene context information. We employ a hidden Markov model (HMM) to model each scene and n-gram language model to represent the contexts among scenes. We evaluated the proposed method in scene recognition experiments for 16 scenes in video data of 25 baseball games. The proposed method significantly improved the results compared to that without scene context information.

REFERENCES

• 1: R. Brunelli, O. Mich, and C. M. Modena, "A survey on the automatic indexing of video data," Journal of Visual Communication and Image Representation, vol. 10, no. 2, pp. 78--112, 1999.
• 2: Sanjiv Kumar , Martial Hebert, A Hierarchical Field Framework for Unified Context-Based Classification, Proceedings of the Tenth IEEE International Conference on Computer Vision, p.1284-1291, October 17-20, 2005 [doi>10.1109/ICCV.2005.9]
• 3: H. B. Nguyen, K. Shinoda, and S. Furui, "Robust highlight extraction using multi-stream hidden Markov models for baseball video," Proc. IEEE International Conference on Image Processing, vol. 3, pp. 173--176, 2005.
• 4: T. Mochizuki, M. Tadenuma, and N. Yagi, "Baseball video indexing using patternization of scenes and hidden Markov model," Proc. IEEE International Conference on Image Processing, vol. 3, pp. 1212--1215, 2005.
• 5: P. Chang, M. Han, and Y. Gong, "Extract highlights from baseball game video with hidden Markov models," Proc. IEEE International Conference on Image Processing, vol.1, pp. I-609--612, 2002.
• 6: Yihong Gong , Mei Han , Wei Hua , Wei Xu, Maximum entropy model-based baseball highlight detection and classification, Computer Vision and Image Understanding, v.96 n.2, p.181-199, November 2004 [doi>10.1016/j.cviu.2004.02.002]
• 7: C.-H. Liang, W.-T. Chu, J.-H. Kuo, J.-L. Wu, and W.-H. Cheng, "Baseball event detection using game-specific feature sets and rules," Proc. IEEE International Symposium on Circuits and Systems, pp. 3829--3832, 2005.
• 8: P. Xu, L. Xie, S. F. Chang, A. Divakaran, A. Vetro, and H. Sun, "Algorithms and system for segmentation and structure analysis in soccer video," Proc. IEEE International Conference on Multimedia and Expo, pp. 928--931, 2001.
• 9: Chua Hock Chuan, Automatic Parsing of TV Soccer Programs, Proceedings of the International Conference on Multimedia Computing and Systems, p.167, May 15-18, 1995 [doi>10.1109/MMCS.1995.484921]
• 10: E. Kijak, L. Oisel, and P. Gros, "Hierarchical structure analysis of sport videos using HMMs," Proc. IEEE International Conference on Image Processing, vol.3, pp. 1025--1028, 2003.
• 11: G. Xu, Y.-F. Ma, H.-J. Zhang, and S.-Q. Yang, "Motion based event recognition using HMM," IEEE Trans. Circuits and Systems, vol. 15, pp. 1422--1433, 2005.
• 12: N. Babaguchi, Y. Kwai, and T. Kitahashi, "Event based indexing of broadcasted sports video by intermodal collaboration," IEEE Trans. Multimedia, vol. 4, no. 1, pp. 68--75, 2002.
• 13: Lawrence Rabiner , Biing-Hwang Juang, Fundamentals of speech recognition, Prentice-Hall, Inc., Upper Saddle River, NJ, 1993
• 14: G. Xu, Y.-F. Ma, H.-J. Zhang, and S. Yang, "Motion based event recognition using HMM," Proc. IEEE International Conference on Pattern Recognition, vol. 2, pp. 831--834, 2002.
• 15: D. Zhong and S. F. Chang, "Structure analysis of sports video using demain models," Proc. IEEE International Conference on Multimedia and Expo, pp. 920--923, 2001.
• 16: S. Takagi , S. Hattori , K. Yokoyama , A. Kodate , H. Tominaga, Sports video categorizing method using camera motion parameters, Proceedings of the 2003 International Conference on Multimedia and Expo, p.461-464, July 06-09, 2003
• 17: B. Lucas and T. Kanade, "An iterative image registration technique with an application to stereo vision," Proc. 7th International Joint Conference on Artificial Intelligence, pp. 674--679, 1981.
• 18: S. M. Katz, "Estimation of probabilities from sparse data for the language model component of a speech recognizer," IEEE Trans. Acoustics, Speech and Signal Processing, vol. 35, pp. 400--401, 1987.
• 19: H. Ney, U. Essen, and R. Kneser, "On structuring probabilistic dependencies in stochastic language modeling," Computer Speech and Language, vol. 8, no. 1, pp. 1--38, 1994.
• 20: P. Placeway, R. Schwartz, P. Fung, and L. Nguyen, "The estimation of powerful language models from small and large corpora," Proc. IEEE Acoustics, Speech and Signal Processing, vol. II, pp. 33--36, 1993.
• 21: I. H. Witten and T. C. Bell, "The zero-frequency problem: Estimating the probabilities of novel events in adaptive text compression," IEEE Trans. Information Theory, vol.37, no. 4, pp. 1085--1094, 1991.
• 22: G. Saon and M. Padmanablan, "Data-driven approach to designing compound words for continuous speech recognition," IEEE Trans. Speech and Audio Processing, vol. 9, no. 4, pp. 327--332, 2001.
• 23: A. Kilgariff and D. Tugwell, "Wasp-bench: an mt lexicographer's workstation supporting state-of-the-art lexical disambiguation," Proc. the 8th Machine Translation Summit, pp. 187--190, 2001.
• 24: http://htk.eng.cam.ac.uk.
• 25: http://svr-www.eng.cam.ac.uk/¿prc14/toolkit.html.
• 26: Xuedong Huang , Hsiao-Wuen Hon , Raj Reddy, Spoken Language Processing: A Guide to Theory, Algorithm, and System Development, Prentice Hall PTR, Upper Saddle River, NJ, 2001
• 27: http://julius.sourceforge.jp.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.10 Vision and Scene Understanding
            Subjects: Video analysis

Additional Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis
          Subjects: Time-varying imagery

General Terms:
Algorithms, Experimentation

Keywords:
CBVIR, HMM, indexing, n-gram model, sports video