Automating camera management for lecture room environments

ABSTRACT

Given rapid improvements in network infrastructure and streaming-media technologies, a large number of corporations and universities are recording lectures and making them available online for anytime, anywhere access. However, producing high-quality lecture videos is still labor intensive and expensive. Fortunately, recent technology advances are making it feasible to build automated camera management systems to capture lectures. In this paper we report on our design, implementation and study of such a system. Compared to previous work-which has tended to be technology centric-we started with interviews with professional video producers and used their knowledge and expertise to create video production rules. We then targeted technology components that allowed us to implement a substantial portion of these rules, including the design of a virtual video director. The system's performance was compared to that of a human operator via a user study. Results suggest that our system's quality in close to that of a human-controlled system. In fact most remote audience members could not tell if the video was produced by a computer or a person.

REFERENCES

• 1: Arijon, D. Grammar of the film language, New York: Communication arts books, Hastings House Publishers, 1976.
• 2: David Bargeron , Anoop Gupta , Jonathan Grudin , Elizabeth Sanocki, Annotations for streaming video on the Web: system design and usage studies, Proceeding of the eighth international conference on World Wide Web, p.1139-1153, May 1999, Toronto, Canada
• 3: Baumberg, A. & Hogg, D., An efficient method for contour tracking using active shape models, TR 94.11, University of Leeds.
• 4: Benesty, J., Adaptive eigenvalue decomposition algorithm for passive acoustic source localization, Journal of Acoustics of America, vol. 107, January 2000, 384-391.
• 5: Bianchi, M., AutoAuditorium: a fully automatic, multi-camera system to televise auditorium presentations, Proc. of Joint DARPA/NIST Smart Spaces Technology Workshop, July 1998.
• 6: Brandstein, M., A pitch-based approach to time delay estimation of reverberant speech, Proc. IEEE ASSP Workshop Appls. Signal Processing Audio Acoustics, 1997.
• 7: Brotherton, J. & Abowd, G., Rooms take note: room takes notes, Proc. AAAI Symposim on Intelligent Environments, 1998, 23-30.
• 8: Buxton, W., Sellen, A., & Sheasby, M., Interfaces for multiparty videoconferences, Video-mediated communication (edited by Finn, K., Sellen, A., & Wilbur, S.), Lawrence Erlbaum Publishers.
• 9: G. Cruz , R. Hill, Capturing and playing multimedia events with STREAMS, Proceedings of the second ACM international conference on Multimedia, p.193-200, October 15-20, 1994, San Francisco, California, United States [doi>10.1145/192593.192654]
• 10: R. Cutler , M. Turk, View-Based Interpretation of Real-Time Optical Flow for Gesture Recognition, Proceedings of the 3rd. International Conference on Face & Gesture Recognition, p.416, April 14-16, 1998
• 11: Li-wei He , Michael F. Cohen , David H. Salesin, The virtual cinematographer: a paradigm for automatic real-time camera control and directing, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.217-224, August 1996 [doi>10.1145/237170.237259]
• 12: Liwei He , Jonathan Grudin , Anoop Gupta, Designing presentations for on-demand viewing, Proceedings of the 2000 ACM conference on Computer supported cooperative work, p.127-134, December 2000, Philadelphia, Pennsylvania, United States [doi>10.1145/358916.358983]
• 13: Gavin Jancke , Jonathan Grudin , Anoop Gupta, Presenting to local and remote audiences: design and use of the TELEP system, Proceedings of the SIGCHI conference on Human factors in computing systems, p.384-391, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332461]
• 14: Jiang, W. & Malvar, H., Adaptive speech noise reduction, Microsoft Research Technical Report, Aug. 1999.
• 15: Sugata Mukhopadhyay , Brian Smith, Passive capture and structuring of lectures, Proceedings of the seventh ACM international conference on Multimedia (Part 1), p.477-487, October 30-November 05, 1999, Orlando, Florida, United States [doi>10.1145/319463.319690]
• 16: Nosa Omoigui , Liwei He , Anoop Gupta , Jonathan Grudin , Elizabeth Sanocki, Time-compression: systems concerns, usage, and benefits, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.136-143, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303017]
• 17: ParkerVision, http://www.parkervision.com/
• 18: PictureTel, http://www.picturetel.com/
• 19: PolyCom, http://www.polycom.com/
• 20: Stanford Online, http://stanford-onlines.stanford.edu/
• 21: Rainer Stiefelhagen , Jie Yang , Alex Waibel, Modeling focus of attention for meeting indexing, Proceedings of the seventh ACM international conference on Multimedia (Part 1), p.3-10, October 30-November 05, 1999, Orlando, Florida, United States [doi>10.1145/319463.319464]
• 22: Wang, C. & Brandstein, M., A hybrid real-time face tracking system, Proc. of ICASSP98, May 1998, Seattle, 3737-3740.
• 23: Shumin Zhai , Carlos Morimoto , Steven Ihde, Manual and gaze input cascaded (MAGIC) pointing, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.246-253, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303053]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Video (e.g., tape, disk, DVI)

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Audio input/output

  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis
          Subjects: Tracking

  K. Computing Milieux
  K.3 COMPUTERS AND EDUCATION

General Terms:
Design, Human Factors, Management, Measurement, Performance, Theory

Keywords:
automated camera management, sound source localization, speaker tracking, video production rules, virtual video director