Using robust audio and video processing technologies to alleviate the elderly cognitive decline

ABSTRACT

We are recently witnessing a growing interest for pervasive context-aware products and services for elderly users. This is largely due to falling fertility and rising longevity phenomena, as well as due to the proliferation of the aging population all over the world. In this paper we present a number of leading edge audio and video processing technologies, which can be exploited to build robust ambient assisted living applications for elderly groups. In particular, we discuss application requirements aiming at alleviating the cognitive decline of elderly users and present audio and video processing components that can essentially fulfill these requirements. We emphasize on technologies such as automatic speech recognition, speaker identification, face detection, person tracking, face identification, and demonstrate how mature versions of these technologies can be appropriately customized to give a significant boost to AAL applications for senior citizens. The challenges, solutions and ideas within this paper are part of the EU project HERMES, which aims at providing an integrated approach to cognitive care, based on assistive technology that reduces age-related decline of cognitive capabilities.

REFERENCES

• 1: Chil (computer in the human interaction loop) eu fp6 integrated project. http://chil.server.de/.
• 2: Hermes eu fp7 specific targeted research project. http://www.fp7-hermes.eu.
• 3: Malach (multilingual access to large spoken archives). http://malach.umiacs.umd.edu/.
• 4: S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp. A tutorial on particle filters for on-line non-linear/non-gaussian bayesian tracking. IEEE Transactions on Signal Processing, 50(2):174--188, Feb. 2002.
• 5: N. Baumgartner and W. Retschitzegger. A survey of upper ontologies for situation awareness. In Knowledge Sharing and Collaborative Engineering (KSCE 2006), St. Thomas, US Virgin Islands, November 2006.
• 6: G. Bradski. Computer vision face tracking for use in a perceptual user interface, 1998.
• 7: R. Brunelli, A. Brutti, P. Chippendale, O. Lanz, M. Omologo, P. Svaizer, and F. Tobia. A generative approach to audio-visual person tracking. In R. Stiefelhagen and J. S. Garofolo, editors, Multimodal Technologies for Perception of Humans, First International Evaluation Workshop on Classification of Events, Activities and Relationships, CLEAR 2006, Southampton, UK, April 6-7, 2006, Revised Selected Papers, volume 4122 of Lecture Notes in Computer Science, pages 55--68. Springer, 2007.
• 8: R. E. Kalman. A new approach to linear filtering and prediction problems. Transactions of the ASME-Journal of Basic Engineering, 82(Series D):35--45, 1960.
• 9: G. Karame, A. Stergiou, N. Katsarakis, P. Papageorgiou, and A. Pnevmatikakis. 2d and 3d face localization for complex scenes. In IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS'07), pages 371--376, London, UK, September 2007. IEEE Computer Society.
• 10: Oswald Lanz, Approximate Bayesian Multibody Tracking, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.28 n.9, p.1436-1449, September 2006 [doi>10.1109/TPAMI.2006.177]
• 11: C. M. Lee, S. S. Narayanan, and R. Pieraccini. Combining acoustic and language information for emotion recognition. In Seventh International Conference on Spoken Language Processing (INTERSPEECH 2002 - ICSLP), pages 873--876, Denver, CO, USA, 2002.
• 12: P. Perez, J. Vermaak, and A. Blake. Data fusion for visual tracking with particles. Proceedings of IEEE, 92(3):495--513, 2004.
• 13: Aristodemos Pnevmatikakis , Lazaros Polymenakos, Robust Estimation of Background for Fixed Cameras, Proceedings of the 15th International Conference on Computing, p.37-42, November 21-24, 2006 [doi>10.1109/CIC.2006.63]
• 14: B. Ramabhadran, O. Siohan, L. Mangu, G. Zweig, M. Westphal, H. Schulz, and A. Soneiro. The ibm 2006 speech transcription system for european parliamentary speeches. In Ninth International Conference on Spoken Language Processing (INTERSPEECH 2006 - ICSLP), Pittsburgh, PA, USA, September 2006.
• 15: E. Rentzeperis, C. Boukis, A. Pnevmatikakis, and L. Polymenakos. Combining finite state machines and lda for voice activity detection. In Artificial Intelligence and Innovations 2007: from Theory to Applications, pages 323--329, Peania, Athens, Greece, September 2007.
• 16: John Soldatos , Nikolaos Dimakis , Kostas Stamatis , Lazaros Polymenakos, A breadboard architecture for pervasive context-aware services in smart spaces: middleware components and prototype applications, Personal and Ubiquitous Computing, v.11 n.3, p.193-212, February 2007 [doi>10.1007/s00779-006-0102-7]
• 17: John Soldatos , Ippokratis Pandis , Kostas Stamatis , Lazaros Polymenakos , James L. Crowley, Agent based middleware infrastructure for autonomous context-aware ubiquitous computing services, Computer Communications, v.30 n.3, p.577-591, February, 2007 [doi>10.1016/j.comcom.2005.11.018]
• 18: Vince Stanford, Pervasive Computing Goes to Work: Interfacing to the Enterprise, IEEE Pervasive Computing, v.1 n.3, p.6-12, July 2002 [doi>10.1109/MPRV.2002.1037716]
• 19: V. M. Stanford. Pervasive computing: Applications - using pervasive computing to deliver elder care. IEEE Distributed Systems Online, 3(3), 2002.
• 20: A. Stergiou, A. Pnevmatikakis, and L. Polymenakos. The ait multimodal person identification system for clear 2007. In CLEAR'07 Evaluation Campaign and Workshop - Classification of Events, Activities and Relationships, Baltimore, MD, USA, May 2007.
• 21: Lawrence D. Stone , Thomas L. Corwin , Carl A. Barlow, Bayesian Multiple Target Tracking, Artech House, Inc., Norwood, MA, 1999
• 22: P. A. Viola and M. J. Jones. Rapid object detection using a boosted cascade of simple features. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2001), pages 511--518, Kauai, HI, USA, December 2001.

INDEX TERMS

Primary Classification:
  J. Computer Applications
  J.3 LIFE AND MEDICAL SCIENCES

Additional Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis

General Terms:
Human Factors

Keywords:
3D tracking, automatic speech, speaker and face recognition