Human-centered multimedia systems: tutorial overview

ABSTRACT

Human-Centered Computing (HCC) is a set of methodologies that apply to any field that uses computers, in any form, in applications in which humans directly interact with devices or systems that use computer technologies. This tutorial takes a holistic view on the research issues and applications of Human-Centered Multimedia Systems focusing on three main areas: (1)multimodal interaction: visual (body, gaze, gesture) and audio (emotion)analysis; (2) image databases, indexing, and retrieval: context modeling, cultural issues, and machine learning for user-centric approaches; (3)multimedia data: conceptual analysis at different levels (feature, cognitive, and affective). This paper gives a brief overview of the areas covered in the tutorial.

REFERENCES

• 1: A. T. Duchowski, "A Breadth-First Survey of Eye Tracking Applications," Behavior Research Methods, Instruments, and Computing, 34(4):455--70, 2002.
• 2: J. Flanagan and T. S. Huang, Special Issue on Human-computer Multimodal Multimodal Interface, Proceedings of the IEEE, 91(9), 2003.
• 3: A. Hanjalic and L-Q. Xu, "Affective video content representation and modeling," IEEE Trans. on Multimedia, 7(1):143--154, 2005.
• 4: Alejandro Jaimes , Nicu Sebe, Multimodal human-computer interaction: A survey, Computer Vision and Image Understanding, v.108 n.1-2, p.116-134, October, 2007 [doi>10.1016/j.cviu.2006.10.019]
• 5: Alejandro Jaimes , Nicu Sebe , Daniel Gatica-Perez, Human-centered computing: a multimedia perspective, Proceedings of the 14th annual ACM international conference on Multimedia, October 23-27, 2006, Santa Barbara, CA, USA [doi>10.1145/1180639.1180829]
• 6: A. Jaimes, D. Gatica-Perez, N. Sebe, and T. S. Huang (eds.) IEEE Computer Magazine special issue on Human-Centered Computing, Vol. 40, No. 5, May 2007.
• 7: A. Jaimes and S.-F. Chang, "A Conceptual Framework for Indexing Visual Information at Multiple Levels", SPIE Internet Imaging, Vol. 3964, pp. 2--15, 2000.
• 8: T. P. Minka and R. W. Picard, "Interactive Learning using a 'Society of Models'," Pattern Recognition, 30(4), 1997.
• 9: M. R. Naphade and T. S. Huang, "Extracting semantics from audio-visual content: the final frontier in multimedia retrieval," IEEE Trans. Neural Networks, 13(4):793--810, 2002.
• 10: V. Pavlovic, A. Garg, and J. M. Rehg, "Boosted learning in dynamic Bayesian networks for multimodal speaker detection," Proceedings of the IEEE, 91(9):1355--1369, 2003.
• 11: Sharon Oviatt , Philip Cohen, Perceptual user interfaces: multimodal interfaces that process what comes naturally, Communications of the ACM, v.43 n.3, p.45-53, March 2000 [doi>10.1145/330534.330538]
• 12: Vladimir I. Pavlovic , Rajeev Sharma , Thomas S. Huang, Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.19 n.7, p.677-695, July 1997 [doi>10.1109/34.598226]
• 13: N. Sebe, I. Cohen, and T. S. Huang, "Multimodal emotion recognition," in Handbook of Pattern Recognition and Computer Vision, C. H. Chen and P. S. P. Wang eds, chapter 4.1, pp. 387--419, World Scientific, January 2005.
• 14: Michael L. Dertouzos, The Unfinished Revolution: Human-Centered Computers and What They Can Do for Us, HarperInformation, 2001
• 15: Mark Weiser, Some computer science issues in ubiquitous computing, Communications of the ACM, v.36 n.7, p.75-84, July 1993 [doi>10.1145/159544.159617]
• 16: Eric Brewer , Michael Demmer , Bowei Du , Melissa Ho , Matthew Kam , Sergiu Nedevschi , Joyojeet Pal , Rabin Patra , Sonesh Surana , Kevin Fall, The Case for Technology in Developing Regions, Computer, v.38 n.6, p.25-38, June 2005 [doi>10.1109/MC.2005.204]
• 17: Ramesh Jain, Folk computing, Communications of the ACM, v.46 n.4, April 2003 [doi>10.1145/641205.641223]
• 18: J. Karat , C. M. Karat, The evolution of user-centered focus in the human-computer interaction field, IBM Systems Journal, v.42 n.4, p.532-541, October 2003
• 19: D. Maynes-Aminzade, R. Pausch, and S. Seitz, "Techniques for interactive audience participation," ICMI 2002

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: User-centered design

Additional Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.9 Applications

General Terms:
Algorithms, Human Factors, Measurement

Keywords:
human-centered computing, human-computer interfaces, multimedia, multimodal interaction