Interactive sound synthesis for large scale environments

ABSTRACT

We present an interactive approach for generating realistic physically-based sounds from rigid-body dynamic simulations. We use spring-mass systems to model each object's local deformation and vibration, which we demonstrate to be an adequate approximation for capturing physical effects such as magnitude of impact forces, location of impact, and rolling sounds. No assumption is made about the mesh connectivity or topology. Surface meshes used for rigid-body dynamic simulation are utilized for sound simulation without any modifications. We use results in auditory perception and a novel priority-based quality scaling scheme to enable the system to meet variable, stringent time constraints in a real-time application, while ensuring minimal reduction in the perceived sound quality. With this approach, we have observed up to an order of magnitude speed-up compared to an implementation without the acceleration. As a result, we are able to simulate moderately complex simulations with upto hundreds of sounding objects at over 100 frames per second (FPS), making this technique well suited for interactive applications like games and virtual environments. Furthermore, we utilize OpenAL and EAX™ on Creative Sound Blaster Audigy 2™ cards for fast hardware-accelerated propagation modeling of the synthesized sound.

REFERENCES

• 1: Chaigne, A., and Doutaut, V. 1997. Numerical simulations of xylophones. i. time domain modeling of the vibrating bars. J. Acoust. Soc. Am. 101, 1, 539--557.
• 2: Chung, J. Y., Liu, J., and Lin, K. J. 1987. Scheduling real-time, periodic jobs using imprecise results. In Proc. IEEE RTS.
• 3: Jack J. Dongarra, Performance of Various Computers Using Standard Linear Equations Software, University of Tennessee, Knoxville, TN, 1989
• 4: Jean-Loup Florens , Claude Cadoz, The physical model: modeling and simulating the instrumental universe, Representations of musical signals, MIT Press, Cambridge, MA, 1991
• 5: Fouad, H., Ballas, J., and Hahn, J. 1997. Perceptually based scheduling algorithms for real-time synthesis of complex sonic environments. In Proc. Int. Conf. Auditory Display.
• 6: Eran Guendelman , Robert Bridson , Ronald Fedkiw, Nonconvex rigid bodies with stacking, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003
• 7: Kim, Y. J., Lin, M. C., and Manocha, D. 2002. DEEP: an incremental algorithm for penetration depth computation between convex polytopes. Proc. of IEEE Conference on Robotics and Automation, 921--926.
• 8: Brian Mirtich , John Canny, Impulse-based simulation of rigid bodies, Proceedings of the 1995 symposium on Interactive 3D graphics, p.181-ff., April 09-12, 1995, Monterey, California, United States [doi>10.1145/199404.199436]
• 9: James F. O'Brien , Perry R. Cook , Georg Essl, Synthesizing sounds from physically based motion, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.529-536, August 2001 [doi>10.1145/383259.383321]
• 10: James F. O'Brien , Chen Shen , Christine M. Gatchalian, Synthesizing sounds from rigid-body simulations, Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation, July 21-22, 2002, San Antonio, Texas [doi>10.1145/545261.545290]
• 11: Sek, A., and Moore. B. C. 1995. Frequency discrimination as a function of frequency, measured in several ways. J. Acoust. Soc. Am. 97, 4 (April), 2479--2486.
• 12: Van Den Doel, K., and Pai, D. K. 1996. Synthesis of shape dependent sounds with physical modeling. In Proceedings of the International Conference on Auditory Displays.
• 13: Van Den Doel, K., and Pai, D. K. 1998. The sounds of physical shapes. Presence 7, 4, 382--395.
• 14: Kees van den Doel , Paul G. Kry , Dinesh K. Pai, FoleyAutomatic: physically-based sound effects for interactive simulation and animation, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.537-544, August 2001 [doi>10.1145/383259.383322]
• 15: Kees van den Doel , Dave Knott , Dinesh K. Pai, Interactive simulation of complex audiovisual scenes, Presence: Teleoperators and Virtual Environments, v.13 n.1, p.99-111, February 2004 [doi>10.1162/105474604774048252]
• 16: Zwicker, E., and Fastl, H. 1990. In Psychoacoustics. Springer-Verlag, Berlin.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.5 Sound and Music Computing

Additional Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.5 Computational Geometry and Object Modeling
          Subjects: Physically based modeling
      I.3.7 Three-Dimensional Graphics and Realism
          Subjects: Animation

Keywords:
OpenAL, rigid-body simulation, sound synthesis