Extending X3D for distributed multimedia processing and control

ABSTRACT

Web-based applications of interactive 3D computer graphics are showing a tendency to get more interconnected and visually complex. Virtual communities like Second Life demand realism not only in terms of realistic rendering, but also in terms of integrated multimedia content. For these Web-based applications, X3D is the ISO-standard way to specify and manipulate scene descriptions. In terms of multimedia integration, however, X3D offers to specify content only in the form of URLs pointing to files. Modern middleware for distributed multimedia, on the other hand, allows applications to harness the full range of multimedia processing as well as transparent use and full control of both local and remote components. Integrating a full multimedia processing pipeline into X3D would enable Web authors to use, for example, streaming media, post-processing on media streams, or routing between scene elements (e.g., sensors) and elements of multimedia processing (e.g., TV cards). A full integration of multimedia in X3D is yet missing.

In this paper, we propose X3D extensions for a seamless mapping of a distributed multimedia flow graph onto an X3D scene graph, making all the features of a distributed multimedia middleware accessible and usable within an X3D scene. We present our proposed specification and implementation of multimedia nodes for X3D. Using examples and implemented X3D application scenarios, we show the simplicity and feasibility of our approach.

REFERENCES

• 1: Apple Inc., 2009. QuickTime Technologies. http://www.apple.com/quicktime/technologies.
• 2: Behr, J., and Dähne, P. 2003. AVALON: Ein komponentenorientiertes Rahmensystem für dynamische Mixed-Reality Anwendungen. In Thema Forschung, 66--73.
• 3: Don Brutzman , Leonard Daly, X3D: Extensible 3D Graphics for Web Authors (The Morgan Kaufmann Series in Interactive 3D Technology) (The Morgan Kaufmann Series in Interactive 3D Technology), Morgan Kaufmann Publishers Inc., San Francisco, CA, 2007
• 4: Brutzman, D., and Kolsch, M. 2007. Video Requirements for Web-based Virtual Environments using Extensible 3D (X3D) Graphics. http://www.w3.org/2007/08/video/positions/Web3D.pdf.
• 5: Creative Labs, 2007. OpenAL. http://openal.org.
• 6: A. Fillinger , L. Diduch , I. Hamchi , M. Hoarau , S. Degre , V. Stanford, The NIST Data Flow System II: A standardized interface for distributed multimedia applications, Proceedings of the 2008 International Symposium on a World of Wireless, Mobile and Multimedia Networks, p.1-3, June 23-26, 2008 [doi>10.1109/WOWMOM.2008.4594879]
• 7: Georgiev, I., Rubinstein, D., Hoffmann, H., and Slusallek, P. 2008. Real Time Ray Tracing on Many-Core-Hardware. In Proceedings of the 5th INTUITION Conference on Virtual Reality.
• 8: ISO/IEC 14496-11, 2007. Scene description and Application engine(BIFS).
• 9: Lohse, M., and Slusallek, P. 2005. Towards Automatic Setup of Distributed Multimedia Applications. In Proceedings of The 9th IASTED International Conference on Internet and Multimedia Systems and Applications (IMSA), ACTA Press, 359--364.
• 10: Marco Lohse , Michael Repplinger , Philipp Slusallek, An Open Middleware Architecture for Network-Integrated Multimedia, Proceedings of the Joint International Workshops on Interactive Distributed Multimedia Systems and Protocols for Multimedia Systems: Protocols and Systems for Interactive Distributed Multimedia, p.327-338, November 26-29, 2002
• 11: Marco Lohse , Florian Winter , Michael Repplinger , Philipp Slusallek, Network-integrated multimedia middleware (NMM), Proceeding of the 16th ACM international conference on Multimedia, October 26-31, 2008, Vancouver, British Columbia, Canada [doi>10.1145/1459359.1459576]
• 12: Chris Marrin , Rob Myers , Jim Kent , Peter Broadwell, Steerable media: interactive television via video synthesis, Proceedings of the sixth international conference on 3D Web technology, p.7-14, February 2001, Paderbon, Germany [doi>10.1145/363361.363365]
• 13: Microsoft, 2009. DirectShow Architecture. http://msdn.microsoft.com/.
• 14: Müller, P., Schubiger-Banz, S., Müller Arisona, S., and Specht, M. 2006. Interactive media and design editing for live visuals applications. In International Conference on Computer Graphics Theory and Applications (GRAPP), 232--241.
• 15: Stefan Müller , Simon Schubiger-Banz , Matthias Specht, A real-time multimedia composition layer, Proceedings of the 1st ACM workshop on Audio and music computing multimedia, October 27-27, 2006, Santa Barbara, California, USA [doi>10.1145/1178723.1178739]
• 16: Repplinger, M., Löffler, A., Thielen, M., and Slusallek, P. 2009. A Flexible Adaptation Service for Distributed Rendering. In Proceedings of Eurographics Parallel Graphics and Visualization Symposium (EGPGV 2009) (to appear).
• 17: Michael Rymaszewski , Wagner James Au , Mark Wallace , Catherine Winters , Cory Ondrejka , Benjamin Batstone-Cunningham , Philip Rosedale, Second Life: The Official Guide, SYBEX Inc., Alameda, CA, 2006
• 18: H. Schulzrinne , S. Casner , R. Frederick , V. Jacobson, RTP: A Transport Protocol for Real-Time Applications, RFC Editor, 2003
• 19: Seibert, H., and Dähne, P. 2006. System Architecture of a Mixed Reality Framework. Journal of Virtual Reality and Broadcasting 3, 7 (May). urn:nbn:de:0009-6-7774, ISSN 1860--2037.
• 20: Michael Still, The Definitive Guide to ImageMagick (Definitive Guide), Apress, Berkely, CA, 2005

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.7 Three-Dimensional Graphics and Realism
          Subjects: Virtual reality

Additional Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.2 Graphics Systems
          Subjects: Distributed/network graphics
      I.3.8 Applications

Keywords:
X3D, distributed and parallel processing, distributed flow graph, distributed processing, multimedia middleware