A multimodal speaker detection and tracking system for teleconferencing

ABSTRACT

A serious problem in both audio and video conferencing facilities available today is the difficulty in determining who is speaking among a large number of participants. There is a strong need for developing meeting room infrastructure and teleconference facilities that improve the sense of presence and participation experienced in remote meetings. We present a distributed multimodal tracking system that uses multiple cameras and microphones to automatically select the current speaker among multiple meeting participants. The system actively obtains and transmits video showing a good view of the selected speaker. The tracking system is integrated into a web-based video conferencing application that connects seven meeting rooms around the globe. An important part of designing such a system is to determine sensor placement and configuration through systematic experiments in the actual rooms where the system is deployed.

REFERENCES

• 1: N. Sukaviriya, B. Yoshimi, H. Derby, B. Bolam, B. Carmelli, J. Elliott, A. Ribak, V. Soroka, and J. Morgan, "Smart Meeting - Support for Sharing Participant Information in Live Meetings through the Web," Proceedings of The Fourth International Conference on Distributed Communities on the Web, Sydney, Australia, 2002.
• 2: D. R. Fischell and C. R. Coker, "A speech direction finder," in Proc. Of International Conference on Acoustics Speech and Signal Processing, 1984, pp. 19.8.1--19.8.4.
• 3: J. Connell, "http://www.research.ibm.com/ecvg/jhc_proj/ faces.html," 2002.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems

General Terms:
Algorithms, Design, Experimentation, Measurement

Keywords:
IP, audio, collaboration, face, localization, telepresence, video