Online audio background determination for complex audio environments

ABSTRACT

We present a method for foreground/background separation of audio using a background modelling technique. The technique models the background in an online, unsupervised, and adaptive fashion, and is designed for application to long term surveillance and monitoring problems. The background is determined using a statistical method to model the states of the audio over time. In addition, three methods are used to increase the accuracy of background modelling in complex audio environments. Such environments can cause the failure of the statistical model to accurately capture the background states. An entropy-based approach is used to unify background representations fragmented over multiple states of the statistical model. The approach successfully unifies such background states, resulting in a more robust background model. We adaptively adjust the number of states considered background according to background complexity, resulting in the more accurate classification of background models. Finally, we use an auxiliary model cache to retain potential background states in the system. This prevents the deletion of such states due to a rapid influx of observed states that can occur for highly dynamic sections of the audio signal. The separation algorithm was successfully applied to a number of audio environments representing monitoring applications.

REFERENCES

• 1: Azlan, M., Cartwright, I., Jones, N., Quirk, T., and West, G. 2005. Multimodal monitoring of the aged in their own homes. In Proceedings of the ICOST'2005: 3rd. International Conference on Smart Homes and Health Telematics (July) Magog, Canada.
• 2: Chen, J., Kam, A. H., Zhang, J., Liu, N., and Shue, L. 2005a. Bathroom activity monitoring based on sound. In Pervasive Computing. Munich, Germany, 47--61.
• 3: Chen, J., Zhang, J., Kam, A., and Shue, L. 2005b. An automatic acoustic bathroom monitoring system. In IEEE International Symposium on Circuits and Systems (ISCAS 05). vol. 2, 1750--1753.
• 4: Clarkson, B., Sawhney, N., and Pentland, A. 1998. Auditory context awareness in wearable computing. In Workshop on Perceptual User Interfaces. San Francisco, U.S.A., 47--61.
• 5: Clavel, C., Ehrette, T., and Richard, G. 2005. Events detection for an audio-based surveillance system. In IEEE International Conference on Multimedia and Expo (ICME 2005). Amsterdam, Netherlands.
• 6: Thomas M. Cover , Joy A. Thomas, Elements of information theory, Wiley-Interscience, New York, NY, 1991
• 7: Michael Cowling , Renate Sitte, Comparison of techniques for environmental sound recognition, Pattern Recognition Letters, v.24 n.15, p.2895-2907, November 2003 [doi>10.1016/S0167-8655(03)00147-8]
• 8: Marco Cristani , Manuele Bicego , Vittorio Murino, On-Line Adaptive Background Modelling for Audio Surveillance, Proceedings of the Pattern Recognition, 17th International Conference on (ICPR'04) Volume 2, p.399-402, August 23-26, 2004 [doi>10.1109/ICPR.2004.653]
• 9: Ingrid Daubechies, Ten lectures on wavelets, Society for Industrial and Applied Mathematics, Philadelphia, PA, 1992
• 10: John R. Deller, Jr. , John G. Proakis , John H. Hansen, Discrete Time Processing of Speech Signals, Prentice Hall PTR, Upper Saddle River, NJ, 1993
• 11: Ahmed M. Elgammal , David Harwood , Larry S. Davis, Non-parametric Model for Background Subtraction, Proceedings of the 6th European Conference on Computer Vision-Part II, p.751-767, June 26-July 01, 2000
• 12: Ellis, D. P. W. 2001. Detecting alarm sounds. In Consistent and Reliable Acoustic Cues for Sound Analysis. Aalborg, Denmark.
• 13: Foote, J. T. and Cooper, M. L. 2003. Media segmentation using self-similarity decomposition. In SPIE Storage and Retrieval for Multimedia Databases. vol. 5021. 167--175.
• 14: Gaunard, P., Mubikangiey, C., Couvreur, C., and Fontaine, V. 1998. Automatic classification of environmental noise events by hidden markov models. In IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '98). vol. 6, 3609--3612.
• 15: Härmä, A., McKinney, M., and Skowronek, J. 2005. Automatic surveillance of the acoustic activity in our living environment. In IEEE International Conference on Multimedia and Expo (ICME 2005). Amsterdam, Netherlands.
• 16: Kim, K., Chalidabhongse, T. H., Harwood, D., and Davis, L. 2004. Background modeling and subtraction by codebook construction. In IEEE International Conference on Image Processing (ICIP). Singapore.
• 17: Lillian Lee, Measures of distributional similarity, Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics, p.25-32, June 20-26, 1999, College Park, Maryland [doi>10.3115/1034678.1034693]
• 18: Moncrieff, S., Venkatesh, S., and West, G. 2005. Persistent audio modelling for background determination. In IEEE International Conference on Multimedia and Expo (ICME 2005). Amsterdam, Netherlands.
• 19: Simon Moncrieff , Svetha Venkatesh , Geoff West, Unifying Background Models over Complex Audio using Entropy, Proceedings of the 18th International Conference on Pattern Recognition, p.249-253, August 20-24, 2006 [doi>10.1109/ICPR.2006.1141]
• 20: Regunathan Radhakrishnan , Ajay Divakaran , Ziyou Xiong, A time series clustering based framework for multimedia mining and summarization using audio features, Proceedings of the 6th ACM SIGMM international workshop on Multimedia information retrieval, October 15-16, 2004, New York, NY, USA [doi>10.1145/1026711.1026738]
• 21: Mathias Stäger , Paul Lukowicz , Niroshan Perera , Thomas von Büren , Gerhard Tröster , Thad Starner, SoundButton: Design of a Low Power Wearable Audio Classification System, Proceedings of the 7th IEEE International Symposium on Wearable Computers, p.12, October 21-23, 2003
• 22: Stauffer, C. and Grimson, W. 1999. Adaptive background mixture models for real-time tracking. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (1999). vol. 2. Fort Collins, CO USA, 246--252.
• 23: Vacher, M., Istrate, D., Besacier, L., Serignat, J. F., and Castelli, E. 2004. Sound detection and classification for medical telesurvey. In 2nd Conference on Biomedical Engineering. ACTA Press, Ed. Innsbruck, Austria, 395--398.
• 24: Ian H. Witten , Eibe Frank, Data mining: practical machine learning tools and techniques with Java implementations, Morgan Kaufmann Publishers Inc., San Francisco, CA, 2000
• 25: Christopher Richard Wren , Ali Azarbayejani , Trevor Darrell , Alex Paul Pentland, Pfinder: Real-Time Tracking of the Human Body, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.19 n.7, p.780-785, July 1997 [doi>10.1109/34.598236]
• 26: Tong Zhang , C.-C. J. Kuo, Hierarchical classification of audio data for archiving and retrieving, Proceedings of the Acoustics, Speech, and Signal Processing, 1999. on 1999 IEEE International Conference, p.3001-3004, March 15-19, 1999 [doi>10.1109/ICASSP.1999.757472]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.1 Models
          Subjects: Statistical

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Indexing methods

General Terms:
Algorithms, Management

Keywords:
Audio analysis, online background modelling, surveillance and monitoring