Interactive virtual studio and immersive televiewer environment

ABSTRACT

In this paper, we propose a novel virtual studio system in which an anchor in the virtual set interacts with televiewers as if they were sharing the same environment. A televiewer participates in the virtual studio environment by sensing and controlling a dummy head equipped with camera, speaker and microphones. The dummy head acts as a surrogate televiewer, providing the viewpoint experienced by the televiewer via a video camera and the sound experienced by the televiewer via microphone in its head. The anchor can not only interact with the virtual set elements but also share the physical studio with the surrogate televiewers. A televiewer with a head-mounted display (HMD) may feel immersed in the virtual studio environment seamlessly combining the virtual set elements with the real studio elements and interact with the anchor and vice versa. The proposed system consists of Interactive Virtual Studio (IVS) environment and Immersive Televiewer Environment (ITE) in which all the physical elements are collected and managed through IVS and the seamlessly mixed virtual and real elements are experienced via ITE. The essential idea is to have a dual universe where what makes a natural interface only physically should remain as physical and what makes easier to represent virtually should remain virtual and these two parallel universe should be coordinated seamlessly to provide the proper mix of the virtual and real mixed reality experience. In practice, this new interactive virtual studio for the immersive tele-meeting environment may be applied to the production of interactive TV program, tele-conferencing, tele-education and others.

REFERENCES

• 1: Michael R. Macedonia , Michael J. Zyda, A Taxonomy for Networked Virtual Environments, IEEE MultiMedia, v.4 n.1, p.48-56, January 1997 [doi>10.1109/93.580395]
• 2: Benford, S. D., Brown, C. C., Reynard, G. T., and Greenhalgh, C. M., in Proceedings of ACM Conference on Computer Supported Cooperative Work (CSCW'96), (Boston, November 1996), ACM Press, 16-20.
• 3: Christian J. Breiteneder , Simon J. Gibbs , Costas Arapis, TELEPORT—an augmented reality teleconferencing environment, Proceedings of the Eurographics workshop on Virtual environments and scientific visualization '96, p.41-49, January 1996, Monte Carlo, Monaco
• 4: Chris Greenhalgh , John Bowers , Graham Walker , John Wyver , Steve Benford , Ian Taylor, Creating a live broadcast from a virtual environment, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.375-384, July 1999 [doi>10.1145/311535.311591]
• 5: Milgram, P., and Kishino, F., A Taxonomy of Mixed Reality Visual Displays, IEICE Transactions on Information and Systems, E77-D, 12, 1321-1329.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Artificial, augmented, and virtual realities

Additional Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.7 Three-Dimensional Graphics and Realism
          Subjects: Virtual reality

General Terms:
Design, Human Factors, Performance, Theory

Keywords:
avatar, interactive TV, interactive virtual studio, mixed reality