Confidence based multimodal fusion for person identification

ABSTRACT

Person identification is of great interest for various kinds of applications and interactive systems. In our system we use face recognition and voice recognition from data recorded in an interactive dialogue system. In such a system, sequential images and sequential utterances can be used to improve recognition accuracy over single hypotheses. The presented approach uses confidence-based fusion for sequence hypotheses, for multimodal fusion, and to provide a reliability measure of the classification quality that can be used to decide when to trust and when to ignore classification results.

REFERENCES

• 1: H. K. Ekenel, M. Fischer, Q. Jin, and R. Stiefelhagen. Multi-modal person identification in a smart environment. CVPR Biometrics Workshop, Minneapolis, USA, June 2007.
• 2: T. Fawcett. Roc graphs: Notes and practical considerations for data mining researchers. Technical report, HPL-2003-4, HP Laboratories, 2003.
• 3: H. Holzapfel, T. Schaaf, H. K. Ekenel, C. Schaaf, and A. Waibel. A robot learns to know people - first contacts of a robot. KI 2006: Advances in Artificial Intelligence, Springer LNCS, 4314, 2007.
• 4: D. Hosmer and S. Lemeshow. Applied Logistic Regression. Wiley, 1989.
• 5: S. Könn, H. Holzapfel, H. K. Ekenel, and A. Waibel. Integrating face-id into an interactive person-id learning system. International Conference on Computer Vision Systems (ICVS'07), Bielefeld, Germany, 2007.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Evaluation/methodology

General Terms:
Experimentation, Reliability

Keywords:
confidence, multimodal fusion, open set person identification