Using wavelets to synthesize stochastic-based sounds for immersive virtual environments

ABSTRACT

Stochastic, or nonpitched, sounds fill our real-world environment. Humans almost continuously hear stochastic sounds, such as wind, rain, motor sounds, and different types of impact sounds. Because of their prevalence in real-world environments, it is important to include these types of sounds for realistic virtual environment simulations. This paper describes a synthesis approach that uses wavelets for modeling stochastic-based sounds. Parameterizations of the wavelet models yield a variety of related sounds from a small set of models. The result is dynamic sound models that can change according to changes in the virtual environment. This paper contains a description of the sound synthesis process, several developed models, and the on-going perceptual experiments for validating the sound synthesis veracity. The developed models and results demonstrate proof of the concept and illustrate the potential of this approach.

REFERENCES

• 1: Ballas, J. 1993. Common factors in the identification of an assortment of brief everyday sounds. Journal of Experimental Psychology: Human Perception and Performance 19, 2, 250--267.
• 2: Cohen, A. and Ryan, R. D. 1995. Wavelets and Multiscale Signal Processing. Chapman & Hall, London.
• 3: Ingrid Daubechies, Ten lectures on wavelets, Society for Industrial and Applied Mathematics, Philadelphia, PA, 1992
• 4: Gaver, W. 1994. Using and creating auditory icons. In Auditory Display: Sonification, Audification, and Auditory Interfaces, G. Kramer, Ed. Vol XVIII. Santa Fe Institute Studies in the Science of Complexity, Addison-Wesley, Reading, MA. 417--446.
• 5: Goldsmith T. E., Johnson, P. J., and Acton, W. H. 1991. Assessing structural knowledge. Journal of Educational Psychology 83, 1, 88--96.
• 6: Miner, N. 1994. Using voice input and audio feedback to enhance the reality of a virtual experience. In Proceedings of the 1994 IMAGE Conference. Tucson, AZ, June 1994.
• 7: Miner, N. 1998. A Wavelet approach to synthesizing perceptually convincing sounds for virtual environments and multi-media. Ph.D. dissertation. University of New Mexico.
• 8: Nadine E. Miner , Thomas P. Caudell, A wavelet synthesis technique for creating realistic virtual environment sounds, Presence: Teleoperators and Virtual Environments, v.11 n.5, p.493-507, October 2002 [doi>10.1162/105474602320935838]
• 9: Mynatt, E. D. 1994. Designing with auditory icons. In Proceedings of the Second International Conference on Auditory Display (ICAD). Nov. 109--119.
• 10: R. Todd Ogden, Essential wavelets for statistical applications and data analysis, Birkhauser Boston Inc., Cambridge, MA, 1996
• 11: Serra, X. 1989. A system for sound analysis/transformation/synthesis based on a deterministic plus stochastic decomposition. Ph.D. Dissertation, Report No. STAN-M-58. CCRMA and Department of Music, Stanford, University, October.
• 12: Shepard, R. N. 1980. Multidimensional scaling, tree-fitting, and clustering. Science 210, 10/24/80, 390--398.
• 13: Smith, J. O. 1992. Physical modeling using digital waveguides. Computer Music Journal, MIT Press 16, 4.
• 14: van den Doel, K. and Pai, D. K. 1997. Synthesis of shape dependent sounds with physical modeling. Proceedings of the International Conference on Auditory Displays. Santa Clara, CA.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.5 Sound and Music Computing

Additional Classification:
  G. Mathematics of Computing
  G.1 NUMERICAL ANALYSIS
      G.1.2 Approximation
          Subjects: Wavelets and fractals

  I. Computing Methodologies
  I.6 SIMULATION AND MODELING
      I.6.4 Model Validation and Analysis

General Terms:
Theory

Keywords:
Sound synthesis, audio perception, immersive environments, virtual reality, wavelets