Building an intelligent camera management system

ABSTRACT

Given rapid improvements in storage devices, network infrastructure and streaming-media technologies, a large number of corporations and universities are recording lectures and making them available online for anytime, anywhere access. However, producing high-quality lecture videos is still labor intensive and expensive. Fortunately, recent technology advances are making it feasible to build automated camera management systems to capture lectures. In this paper we report our design of such a system, including system configuration, audio-visual tracking techniques, software architecture, and user study. Motivated by different roles in a professional video production team, we have developed a multi-cinematographer single-director camera management system. The system performs lecturer tracking, audience tracking, and video editing all fully automatically, and offers quality close to that of human-operated systems.

REFERENCES

• 1: Baumberg, A. & Hogg, D., An efficient method for contour tracking using active shape models, TR 94.1 I, Univ. of Leeds.
• 2: Benesty, J., Adaptive eigenvalue decomposition algorithm for passive acoustic source localization, Journal of Acoustics of America, vol. 107, January 2000,384-39 I.
• 3: Bianchi, M., AutoAuditorium: a fully automatic, multi-camera system to televise auditorium presentations, Proc. qf Joint DARPA/NISTSmart Spaces Technology Workshop, July 1998.
• 4: Brandstein, M. and Silverman, H., "A Practical Methodology for Speech Source Localization with Microphone Arrays," Computer, Speech, and Language, 1 I (2):9 l- 126, April 1997.
• 5: Brotherton, J. & Abowd, G., Rooms take note: room takes notes!, Proc. AAAI Symposim on Intelligent Environments, 1998,23-30.
• 6: Buxton, W., Sellen, A., & Sheasby, M., Interfaces for multiparty videoconferences, Video-mediated communication (edited by Finn, K., Sellen, A., & Wilbur, S.), Lawrence Erlbaum Publishers.
• 7: G. Cruz , R. Hill, Capturing and playing multimedia events with STREAMS, Proceedings of the second ACM international conference on Multimedia, p.193-200, October 15-20, 1994, San Francisco, California, United States [doi>10.1145/192593.192654]
• 8: R. Cutler , M. Turk, View-Based Interpretation of Real-Time Optical Flow for Gesture Recognition, Proceedings of the 3rd. International Conference on Face & Gesture Recognition, p.416, April 14-16, 1998
• 9: Jonathan Foote , Don Kimber, FlyCam: practical panoramic video, Proceedings of the eighth ACM international conference on Multimedia, p.487-488, October 2000, Marina del Rey, California, United States [doi>10.1145/354384.376401]
• 10: Michael Gleicher , James Masanz, Towards virtual videography (poster session), Proceedings of the eighth ACM international conference on Multimedia, p.375-378, October 2000, Marina del Rey, California, United States [doi>10.1145/354384.354537]
• 11: Li-wei He , Michael F. Cohen , David H. Salesin, The virtual cinematographer: a paradigm for automatic real-time camera control and directing, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.217-224, August 1996 [doi>10.1145/237170.237259]
• 12: Liwei He , Jonathan Grudin , Anoop Gupta, Designing presentations for on-demand viewing, Proceedings of the 2000 ACM conference on Computer supported cooperative work, p.127-134, December 2000, Philadelphia, Pennsylvania, United States [doi>10.1145/358916.358983]
• 13: Qiong Liu , Yong Rui , Anoop Gupta , J. J. Cadiz, Automating camera management for lecture room environments, Proceedings of the SIGCHI conference on Human factors in computing systems, p.442-449, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365310]
• 14: Sugata Mukhopadhyay , Brian Smith, Passive capture and structuring of lectures, Proceedings of the seventh ACM international conference on Multimedia (Part 1), p.477-487, October 30-November 05, 1999, Orlando, Florida, United States [doi>10.1145/319463.319690]
• 15: ParkerVision, http://www.parkervision.com/
• 16: PictureTel, http:Nwww.picturetel.com/
• 17: PolyCom, http://www.polycom.com/
• 18: Sound Professionals, http://www.soundprofessionals.com/ moreinfopages/cardioid/generalinfo.html
• 19: Stanford Online, http://stanford-onlines.stanford.edu/
• 20: Rainer Stiefelhagen , Jie Yang , Alex Waibel, Modeling focus of attention for meeting indexing, Proceedings of the seventh ACM international conference on Multimedia (Part 1), p.3-10, October 30-November 05, 1999, Orlando, Florida, United States [doi>10.1145/319463.319464]
• 21: Wang, C. & Brandstein, M., A hybrid real-time face tracking system, Proc. of ICASSP98, May 1998, Seattle, 3737-3740.
• 22: H. Wang , P. Chu, Voice Source Localization for Automatic Camera Pointing System in Videoconferencing, Proceedings of the 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '97) -Volume 1, p.187, April 21-24, 1997
• 23: Zhengyou Zhang, A Flexible New Technique for Camera Calibration, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.22 n.11, p.1330-1334, November 2000 [doi>10.1109/34.888718]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Evaluation/methodology

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Artificial, augmented, and virtual realities

  K. Computing Milieux
  K.3 COMPUTERS AND EDUCATION
      K.3.1 Computer Uses in Education
          Subjects: Collaborative learning

General Terms:
Design, Experimentation, Performance

Keywords:
automated camera management, lecturer tracking, sound source localization, virtual cinematographer, virtual director