BiReality: mutually-immersive telepresence

ABSTRACT

BiReality (a.k.a. Mutually-Immersive Telepresence) uses a teleoperated robotic surrogate to provide an immersive telepresence system for face-to-face interactions. Our goal is to recreate to the greatest extent practical, both for the user and the people at the remote location, the sensory experience relevant for face-to-face interactions of the user actually being in the remote location. Our system provides a 360-degree surround immersive audio and visual experience for both the user and remote participants, and streams eight 704x480 MPEG-2 coded videos totaling 20Mb/s. The system preserves gaze and eye contact, presents local and remote participants to each other at life size, and preserves the head height of the user at the remote location. Initial user experiences are presented.

REFERENCES

• 1: J. Augustine, S. R. Kokrady., N. P. Jouppi, and S. Iyer. Region of Interest Editing of MPEG-2 Video Streams in the Compressed Domain. In Proc. of IEEE International Conference on Multimedia and Expo, 2004.
• 2: H. Harlyn Baker , Nina Bhatti , Donald Tanguay , Irwin Sobel , Dan Gelb , Michael E. Goss , John MacCormick , Kei Yuasa , W. Bruce Culbertson , Thomas Malzbender, Computation and performance issues In coliseum: an immersive videoconferencing system, Proceedings of the eleventh ACM international conference on Multimedia, November 02-08, 2003, Berkeley, CA, USA [doi>10.1145/957013.957118]
• 3: J. Blauert. Spatial Hearing: The Psychophysics of Human Sound Localization. MIT Press, 2nd edition, 1997.
• 4: Bill Buxton , Tom Moran, EuroPARC's integrated interactive intermedia facility (IIIF): early experiences, Proceedings of the IFIP WG 8.4 confernece on Multi-user interfaces and applications, p.11-34, January 1990, Heraklion, Crete, Greece
• 5: William A. S. Buxton, Telepresence: integrating shared task and person spaces, Proceedings of the conference on Graphics interface '92, p.123-129, September 1992, Vancouver, British Columbia, Canada
• 6: Milton Chen, A low-latency lip-synchronized videoconferencing system, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642692]
• 7: Milton Chen, Leveraging the asymmetric sensitivity of eye contact for videoconference, Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/503376.503386]
• 8: E. T. Hall. The Hidden Dimension. Anchor Press, 1990.
• 9: Norman P. Jouppi, First steps towards mutually-immersive mobile telepresence, Proceedings of the 2002 ACM conference on Computer supported cooperative work, November 16-20, 2002, New Orleans, Louisiana, USA [doi>10.1145/587078.587128]
• 10: N. P. Jouppi and S. Iyer. A Headphone-free Head-tracked Audio System for Advanced Audio Telepresence. In Proc. the 117th Audio Engineering Society Convention, October 2004.
• 11: M. W. Matlin and H. J. Foley. Sensation and Perception. Allyn and Bacon, 4th edition, 1997.
• 12: Hideyuki Nakanishi , Chikara Yoshida , Toshikazu Nishimura , Toru Ishida, FreeWalk: A 3D Virtual Space for Casual Meetings, IEEE MultiMedia, v.6 n.2, p.20-28, April 1999 [doi>10.1109/93.771370]
• 13: B. O'Conaill, S. Whittaker, and S. Wilbur. Conversations Over Video Conferences: An Evaluation of the Spoken Aspects of Video-Mediated Communication. Human-Computer Interaction, 8:389--428, 1993.
• 14: Ken-Ichi Okada , Fumihiko Maeda , Yusuke Ichikawaa , Yutaka Matsushita, Multiparty videoconferencing at virtual social distance: MAJIC design, Proceedings of the 1994 ACM conference on Computer supported cooperative work, p.385-393, October 22-26, 1994, Chapel Hill, North Carolina, United States [doi>10.1145/192844.193054]
• 15: Eric Paulos , John Canny, PRoP: personal roving presence, Proceedings of the SIGCHI conference on Human factors in computing systems, p.296-303, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274686]
• 16: A. Prussog, L. Mühlbach, and M. Böcker. Telepresence in Videocommunications. In Proc. the Human Factors and Ergonomics Society 38th Annual Meeting, pages 180--184, 1994.
• 17: A. Sellen. Remote conversations: The effect of mediating talk with technology. Human-Computer Interaction, 10:401--444, 1995.
• 18: Abigail Sellen , Bill Buxton , John Arnott, Using spatial cues to improve videoconferencing, Proceedings of the SIGCHI conference on Human factors in computing systems, p.651-652, May 03-07, 1992, Monterey, California, United States [doi>10.1145/142750.143070]
• 19: Roel Vertegaal, The GAZE groupware system: mediating joint attention in multiparty communication and collaboration, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.294-301, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303065]
• 20: Roel Vertegaal , Ivo Weevers , Changuk Sohn , Chris Cheung, GAZE-2: conveying eye contact in group video conferencing using eye-controlled camera direction, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642702]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.5 Sound and Music Computing

  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.2 Graphics Systems

General Terms:
Design, Experimentation, Human Factors

Keywords:
audio conferencing, computer-supported collaborative work, robotics, spatial audio, video conferencing, video mediated communication