Inter-stream synchronization between haptic media and voice in collaborative virtual environments

ABSTRACT

This paper addresses an inter-stream synchronization issue between haptic media and voice in networked virtual environments. The paper proposes a media synchronization algorithm for the two types of media stream by enhancing the virtual-time rendering (VTR) algorithm, which the authors previously proposed. The new algorithm employs two types of error range in order to keep high quality of intra-stream synchronization at the expense of slight deterioration in the inter-stream synchronization quality. One is the imperceptible range of inter-stream synchronization error, and the other is the allowable range of the error. If the error is beyond the allowable range, the algorithm tries to reduce the error gradually until the error becomes within the imperceptible range. Otherwise, it accepts the error. By carrying out an experiment in which two users do collaborative work with haptic media and voice in a virtual space, we demonstrate the effectiveness of the algorithm quantitatively.

REFERENCES

• 1: M. A. Srinivasan and C. Basdogan, "Haptics in virtual environments: Taxonomy, research status, and challenges," Computers and Graphics, vol. 21, no. 4, pp. 393--404, Apr. 1997.
• 2: G. Blakowski and R. Steinmetz, "A media synchronization survey: Reference model, specification, and case studies," IEEE J. Sel. Areas in Commun., vol. 14, no. 1, pp. 5--35, Jan. 1996.
• 3: K. Hikichi, H. Morino, Y. Yasuda, I. Arimoto, and K. Sezaki, "The evaluation of adaptive control for haptics collaboration over the Internet," in Proc. the 16th International Workshop on Communications Quality & Reliability (CQR'02), pp. 218--222, May 2002 (also Conf. Rec. IEEE GLOBECOM'02, Nov. 2002).
• 4: Y. Ishibashi, S. Tasaka, and T. Hasegawa, "The virtual-time rendering algorithm for haptic media synchronization in networked virtual environments," in Proc. the 16th International Workshop on Communications Quality & Reliability (CQR'02), pp. 213--217, May 2002.
• 5: I. Elhajj, N. Xi, B. Song, M.-M. Yu, W. T. Lo, and Y. H. Liu, "Transparency and synchronization in supermedia enhanced Internet-based teleoperation," in Proc. the 2002 IEEE International Conference on Robotics & Automation, pp. 2713--2718, May 2002.
• 6: Y. Ishibashi , S. Tasaka, A synchronization mechanism for continuous media in multimedia communications, Proceedings of the Fourteenth Annual Joint Conference of the IEEE Computer and Communication Societies (Vol. 3)-Volume, p.1010, April 02-06, 1995
• 7: Y. Ishibashi, T. Hasegawa, and S. Tasaka, "Group synchronization control for haptic media in networked virtual environments," in Proc. the 12th IEEE Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (Haptics'04), pp. 106--113, Mar. 2004.
• 8: T. Harada, S. Ohno, and M. Sato, "The influence of multimodal sensory information display on dribbling of a basketball in a virtual workspace," in Proc. the 4th International Conference on Virtual Systems and Multimedia, vol. 2, pp. 548--553, 1998.
• 9: D. E. DiFranco, G. L. Beauregard, and M. A. Srinivasan, "The effect of auditory cues on the haptic perception of stiffness in virtual environments," in Proc. the ASME Dynamic Systems and Control Division, vol. 61, pp. 17--22, 1997.
• 10: S. Matsumoto, I. Fukuda, H. Morino, K. Hikichi, K. Sezaki, and Y. Yasuda, "The influences of network issues on haptic collaboration in shared virtual environments," in Proc. the Fifth PHANToM Users Group Workshop, Oct. 2000.
• 11: ITU-T Recommendation G.114, "Transmission systems and media, general characteristics of international telephone connections and international telephone circuits: One-way transmission time," Feb. 1996.
• 12: J. Kenneth Salisbury , Mandayam A. Srinivasan, Phantom-Based Haptic Interaction with Virtual Objects, IEEE Computer Graphics and Applications, v.17 n.5, p.6-10, September 1997 [doi>10.1109/MCG.1997.1626171]
• 13: SensAble Technologies, Inc., "GHOST SDK programmer's guide," Version 3.0, 1999.
• 14: Y. Ishibashi, S. Tasaka, and Y. Tachibana, "Media synchronization and causality control for distributed multimedia applications," IEICE Trans. Commun., vol. E84-B, no. 3, pp. 667--677, Mar. 2001.
• 15: R. Steinmetz, "Human perception of jitter and media synchronization," IEEE J. Sel. Areas in Commun., vol. 14, no. 1, pp. 61--72, Jan. 1996.
• 16: S. Tasaka, T. Nunome, and Y. Ishibashi, "Live media synchronization quality of a retransmission-based error recovery scheme," in Conf. Rec. IEEE ICC'00, pp. 1535--1541, June 2000.
• 17: S. Deering, "Host extension for IP multicasting," RFC 1112, Aug. 1989.
• 18: Yang-hua Chu , Sanjay G. Rao , Hui Zhang, A case for end system multicast (keynote address), Proceedings of the 2000 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, p.1-12, June 18-21, 2000, Santa Clara, California, United States

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.4 INFORMATION SYSTEMS APPLICATIONS
      H.4.3 Communications Applications

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.3 Group and Organization Interfaces
          Subjects: Computer-supported cooperative work

General Terms:
Algorithms, Experimentation, Human Factors, Performance

Keywords:
collaborative virtual environments, haptic media, media synchronization, voice