Post-recording tool for instant casting movie system

ABSTRACT

This paper proposes a universal user-friendly post-recording tool for an Instant Casting Movie System (ICS) that enables anyone to be a movie star using his or her own voice and faces. A personal CG character is automatically generated by scanning one's face geometry and image in ICS. Voice is as essential to identify a person as face. However, a character's voice is only based on gender in ICS. We proposed a novel voice recording tool for participants of all ages in a short time. Post-recording tasks are very difficult because speakers should speak in synchronization with the mouth movements of the CG characters. Therefore this task is generally recorded by professional voice actors. Our proposed tool has the following four features: 1) various supporting information for synchronization with voice and mouth movement timing for users; 2) automatic post-processing of recorded voices for compositing mixed audio; 3) intuitively displays operation for people of all ages; and 4) handles multiple users in parallel for quick recording. We developed a prototype speech synchronization system using a post-recording tool and conducted subjective evaluation experiments of it. Over 60% of the subjects responded that the tool's interface can be operated easily.

REFERENCES

• 1: Maejima, A., Wemler, S., Machida, T., Takebayahashi, M., and Morishima, S. 2008. Instant Casting Movie Theater: The Future Cast System. The IEICE Transaction on Information and System, Vol.E91-D, No.4, pp.1135--1148.
• 2: Zen, H., Nose, T., Yamagishi, J., Sako, S., Masuko, T., Black, A.W., Tokuda, K. 2007. The HMM-based speech synthesis system version 2.0, Proc. of ISCA SSW6, Bonn, Germany.
• 3: Kawai, H., Toda, T., Yamagishi, J., Hirai, T., Ni, J., Nishizawa, N., Tsuzaki, M., Tokuda, K. 2006. XIMERA: a concatenative speech synthesis system with large scale corpora. IEICE Trans. J89-D-II, no.12, pp.2688--2698.
• 4: A. J. Hunt , A. W. Black, Unit selection in a concatenative speech synthesis system using a large speech database, Proceedings of the Acoustics, Speech, and Signal Processing, 1996. on Conference Proceedings., 1996 IEEE International Conference, p.373-376, May 07-10, 1996 [doi>10.1109/ICASSP.1996.541110]
• 5: Robert A. J. Clark , Korin Richmond , Simon King, Multisyn: Open-domain unit selection for the Festival speech synthesis system, Speech Communication, v.49 n.4, p.317-330, April, 2007 [doi>10.1016/j.specom.2007.01.014]

INDEX TERMS

Primary Classification:
  J. Computer Applications
  J.5 ARTS AND HUMANITIES

Additional Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.2 Graphics Systems

General Terms:
Human Factors

Keywords:
instant casting movie system, voice post-recording