Hierarchical voting classification scheme for improving visual sign language recognition

ABSTRACT

As one of the important research areas of multimodal interaction, sign language recognition (SLR) has attracted increasing interest. In SLR, especially on medium or large vocabulary, it is usually difficult or impractical to collect enough training data. Thus, how to improve the recognition on the limited training samples is a significant issue. In this paper, a simple but effective hierarchical voting classification (HVC) scheme for improving visual SLR, which makes efficient use of limited training data, is proposed. The key idea of HVC scheme is similar to but not the same as Bagging technique. Firstly, it constructs several training sets from the original training set in a combinatorial fashion to generate the corresponding continuous hidden Markov models (CHMM) ensemble. Then, it determines the ensemble output by appropriate local voting strategy. Finally, it obtains the final recognition result by the global voting. Experimental results show that the HVC scheme outperforms the conventional single CHMM approach in terms of recognition accuracy on the limited training data.

REFERENCES

• 1: R. -H. Liang , M. Ouhyoung, A Real-Time Continuous Gesture Recognition System for Sign Language, Proceedings of the 3rd. International Conference on Face & Gesture Recognition, p.558, April 14-16, 1998
• 2: W. Gao, J.Y. Ma, J.Q. Wu, and C.L. Wang, Sign language recognition based on HMM/ANN/DP, Int'l J. PRAI, vol. 14, no. 5, 2000, 587--602.
• 3: Thad Starner , Alex Pentland , Joshua Weaver, Real-Time American Sign Language Recognition Using Desk and Wearable Computer Based Video, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.20 n.12, p.1371-1375, December 1998 [doi>10.1109/34.735811]
• 4: C. Vogler and D. Metaxas, Adapting hidden Markov models for ASL recognition by using three-dimensional computer vision methods, In Proc. IEEE SMC, 1997, 156--161.
• 5: Christian Vogler , Dimitris Metaxas, A framework for recognizing the simultaneous aspects of American sign language, Computer Vision and Image Understanding, v.81 n.3, p.358-384, March 2001 [doi>10.1006/cviu.2000.0895]
• 6: K. Grobel and M. Assan, Isolated sign language recognition using hidden Markov models, In Proc. SMC, 1996, 162--167.
• 7: Liang-Guo Zhang , Yiqiang Chen , Gaolin Fang , Xilin Chen , Wen Gao, A vision-based sign language recognition system using tied-mixture density HMM, Proceedings of the 6th international conference on Multimodal interfaces, October 13-15, 2004, State College, PA, USA [doi>10.1145/1027933.1027967]
• 8: Eric Bauer , Ron Kohavi, An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants, Machine Learning, v.36 n.1-2, p.105-139, July-August 1999 [doi>10.1023/A:1007515423169]
• 9: L. R. Rabiner, A tutorial on hidden Markov models and selected applications in speech recognition, Proceedings of the IEEE, vol. 77, no. 2, 1989, 257--285.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human information processing

Additional Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.2 Design Methodology
          Subjects: Classifier design and evaluation

General Terms:
Algorithms, Design, Experimentation, Human Factors, Languages

Keywords:
classifiers ensemble, hidden Markov models, hierarchical voting classification, sign language recognition