TAPESTREA: a new way to design sound

ABSTRACT

TAPESTREA is a sound design and composition framework that facilitates the creation of new sound from existing digital audio recordings, through interactive analysis, transformation and re-synthesis. During analysis, sound templates of diffierent types are extracted using a variety of techniques. Each extracted template is transformed and synthesized independently, allowing specialized transformations on each template based on its type. The user interacts with TAPESTREA via a set of graphical interfaces that offer parametric control over every stage of analysis, transformation and re-synthesis. Synthesis is further controlled through ChucK scripts. These combined techniques form a workbench for completely transforming a sound scene, dynamically generating soundscapes, or creating musical tapestries by weaving together transformed elements from different recordings. Thus, TAPESTREA introduces a new paradigm for sound design, composition and sonic sculpting tasks.

REFERENCES

• 1: X. Amatriain and P. Arumi. Developing cross-platform audio and music applications with the CLAM framework. In Proceedings of the International Computer Music Conference, 2005.
• 2: M. Athineos and D. P. W. Ellis. Sound texture modeling with linear prediction in both time and frequency domains. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, volume 5, pages 648--651, 2003.
• 3: J. P. Bello, L. Daudet, S. Abdallah, C. Duxbury, M. Davies, and M. B. Sandler. A tutorial on onset detection in music signals. IEEE Transactions on Speech and Audio Processing, 13(5), 2005.
• 4: N. Bogaards, A. Robel, and X. Rodet. Sound analysis and processing with AudioSculpt 2. In Proceedings of the International Computer Music Conference, 2004.
• 5: M. B. Dolson. The phase vocoder: A tutorial. Computer Music Journal, 10(4):14--27, 1986.
• 6: S. Dubnov, Z. Bar-Joseph, R. El-Yaniv, D. Lischinski, and M. Werman. Synthesizing sound textures through wavelet tree learning. IEEE Computer Graphics and Applications, 22(4), 2002.
• 7: K. Fitz, L. Haken, S. Lefvert, and M. O'Donnel. Sound morphing using Loris and the reassigned bandwidth-enhanced additive sound model: Practice and applications. In Proceedings of the International Computer Music Conference, 2002.
• 8: M. Frojd and A. Horner. Fast sound texture synthesis using overlap-add. In Proceedings of the International Computer Music Conference, 2007.
• 9: M. Klingbeil. Software for spectral analysis, editing, and synthesis. In Proceedings of the International Computer Music Conference, 2005.
• 10: T. Lieber, A. Misra, and P. R. Cook. Freedom in TAPESTREA! Voice-aware track manipulations. In Proceedings of the International Computer Music Conference, 2008.
• 11: R. McAulay and T. Quatieri. Speech analysis/synthesis based on a sinusoidal representation. IEEE Transactions on Acoustics, Speech and Signal Processing, 34(4):744--754, 1986.
• 12: A. Misra, P. R. Cook, and G. Wang. A new paradigm for sound design. In Proceedings of the International Conference on Digital Audio E ects (DAFx), 2006.
• 13: A. Misra, G. Wang, and P. R. Cook. Musical tapestries: Re-composing natural sounds. Journal of New Music Research, 36(4), 2007.
• 14: T. Quatieri and R. McAulay. Speech transformations based on a sinusoidal representation. IEEE Transactions on Acoustics, Speech and Signal Processing, 34(6):1449--1464, 1986.
• 15: P. Schae er. A la Recherche d'une Musique Concrete. Seuil, Paris, 1952.
• 16: X. Serra. A System for Sound Analysis / Transformation / Synthesis based on a Deterministic plus Stochastic Decomposition. PhD thesis, Stanford University, 1989.
• 17: T. S. Verma and T. H. Meng. An analysis/synthesis tool for transient signals that allows a flexible sines+transients+noise model for audio. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, pages 12--15, 1998.
• 18: G. Wang and P. R. Cook. ChucK: A concurrent, on-the-fly, audio programming language. In Proceedings of the International Computer Music Conference, 2003.
• 19: X. Zhu and L. Wyse. Sound texture modeling and time-frequency LPC. In Proceedings of the International Conference on Digital Audio Eff ects (DAFx), 2004.

INDEX TERMS

Primary Classification:
  D. Software
  D.0 GENERAL

General Terms:
Design, Experimentation

Keywords:
audio, composition, multimedia, open source software, real-time, signal processing, sound design