Modeling human interaction resources to support the design of wearable multimodal systems

ABSTRACT

Designing wearable application interfaces that integrate well into real-world processes like aircraft maintenance or medical examinations is challenging. One of themain success criteria is how well the multimodal interaction with the computer system fits an already existing real-world task. Therefore, the interface design needs to take the real-world task flow into account from the beginning.

We propose a model of interaction devices and human interaction capabilities that helps evaluate how well different interaction devices/techniques integrate with specific real-world scenarios. The model was developed based on a survey of wearable interaction research literature. Combining this model with descriptions of observed real-world tasks, possible conflicts between task performance and device requirements can be visualized helping the interface designer to find a suitable solution.

REFERENCES

• 1: Michel Beaudouin-Lafon, Instrumental interaction: an interaction model for designing post-WIMP user interfaces, Proceedings of the SIGCHI conference on Human factors in computing systems, p.446-453, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332473]
• 2: Teresa W. Bleser , John Sibert, Toto: a tool for selecting interaction techniques, Proceedings of the 3rd annual ACM SIGGRAPH symposium on User interface software and technology, p.135-142, October 03-05, 1990, Snowbird, Utah, United States [doi>10.1145/97924.97940]
• 3: Michael Boronowsky , Tom Nicolai , Christoph Schlieder , Ansgar Schmidt, Winspect - A Case Study for Wearable Computing Supported Inspection Tasks, Proceedings of the 5th IEEE International Symposium on Wearable Computers, p.163, October 08-09, 2001
• 4: C. Bürgy. An Interaction Constraints Model for Mobile and Wearable Computer-Aided Engineering Systems in Industrial Applications. PhD thesis, Carnegie Mellon University, 2002.
• 5: William Buxton, Lexical and pragmatic considerations of input structures, ACM SIGGRAPH Computer Graphics, v.17 n.1, p.31-37, January 1983 [doi>10.1145/988584.988586]
• 6: Stuart K. Card , Jock D. Mackinlay , George G. Robertson, The design space of input devices, Proceedings of the SIGCHI conference on Human factors in computing systems: Empowering people, p.117-124, April 01-05, 1990, Seattle, Washington, United States [doi>10.1145/97243.97263]
• 7: Stuart K. Card , Jock D. Mackinlay , George G. Robertson, A morphological analysis of the design space of input devices, ACM Transactions on Information Systems (TOIS), v.9 n.2, p.99-122, April 1991 [doi>10.1145/123078.128726]
• 8: Lucy E. Dunne , Barry Smyth, Psychophysical elements of wearability, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240674]
• 9: Masaaki FUKUMOTO, A Finger-Ring ShapedWearable HANDset based on Bone-Conduction, Proceedings of the Ninth IEEE International Symposium on Wearable Computers, p.10-13, October 18-21, 2005 [doi>10.1109/ISWC.2005.4]
• 10: F. Gemperle , C. Kasabach , J. Stivoric , M. Bauer , R. Martin, Design for Wearability, Proceedings of the 2nd IEEE International Symposium on Wearable Computers, p.116, October 19-20, 1998
• 11: Bruce Howard , Susie Howard, Lightglove: Wrist-Worn Virtual Typing and Pointing, Proceedings of the 5th IEEE International Symposium on Wearable Computers, p.172, October 08-09, 2001
• 12: A. Jameson and K. Klöckner. User multitasking with mobile multimodal systems. In W. Minker, D. Bühler, and L. Dybkjær, editors, Spoken Multimodal Human-Computer Dialogue in Mobile Environments. Kluwer Academic Publishers, Dordrecht, 2004.
• 13: Tobias Klug, Computer aided observations of complex mobile situations, CHI '07 extended abstracts on Human factors in computing systems, April 28-May 03, 2007, San Jose, CA, USA [doi>10.1145/1240866.1241032]
• 14: James S. Lipscomb , Michael E. Pique, Analog input device physical characteristics, ACM SIGCHI Bulletin, v.25 n.3, p.40-45, July 1993 [doi>10.1145/155786.155794]
• 15: Yili Liu , Robert Feyen , Omer Tsimhoni, Queueing Network-Model Human Processor (QN-MHP): A computational architecture for multitask performance in human-machine systems, ACM Transactions on Computer-Human Interaction (TOCHI), v.13 n.1, p.37-70, March 2006 [doi>10.1145/1143518.1143520]
• 16: Christian Metzger , Matt Anderson , Thad Starner, FreeDigiter: A Contact-Free Device for Gesture Control, Proceedings of the Eighth International Symposium on Wearable Computers, p.18-21, October 31-November 03, 2004 [doi>10.1109/ISWC.2004.23]
• 17: Haruo Noma , Aki Ohmura , Noriaki Kuwahara , Kiyoshi Kogure, Wearable Sensors for Auto-Event-Recording on Medical Nursing - User Study of Ergonomic Design -, Proceedings of the Eighth International Symposium on Wearable Computers, p.8-15, October 31-November 03, 2004 [doi>10.1109/ISWC.2004.48]
• 18: John Kangchun Perng , Brian Fisher , Seth Hollar , Kristofer S. J. Pister, Acceleration Sensing Glove, Proceedings of the 3rd IEEE International Symposium on Wearable Computers, p.178, October 18-19, 1999
• 19: Jun Rekimoto, GestureWrist and GesturePad: Unobtrusive Wearable Interaction Devices, Proceedings of the 5th IEEE International Symposium on Wearable Computers, p.21, October 08-09, 2001
• 20: Thad Eugene Starner , Alex Pentland, Wearable computing and contextual awareness, Massachusetts Institute of Technology, 1999
• 21: Maribeth Gandy , Thad Starner , Jake Auxier , Daniel Ashbrook, The Gesture Pendant: A Self-illuminating, Wearable, Infrared Computer Vision System for Home Automation Control and Medical Monitoring, Proceedings of the 4th IEEE International Symposium on Wearable Computers, p.87, October 18-21, 2000
• 22: Bruce Thomas , Karen Grimmer , Dan Makovec , Joanne Zucco , Bernard Gunther, Determination of Placement of a Body-Attached Mouse as a Pointing Input Device for Wearable Computers, Proceedings of the 3rd IEEE International Symposium on Wearable Computers, p.193, October 18-19, 1999
• 23: B. Thomas, S. Tyerman, and K. Grimmer. Evaluation of text input mechanisms for wearable computers. Virtual Reality, 3(3):187--199, 1998.
• 24: Aaron Toney , Lucy Dunne , Bruce H. Thomas , Susan P. Ashdown, A Shoulder Pad Insert Vibrotactile Display, Proceedings of the 7th IEEE International Symposium on Wearable Computers, p.35, October 21-23, 2003
• 25: B. H. Thomas, Minimal Social Weight User Interactions for Wearable Computers in Business Suits, Proceedings of the 6th IEEE International Symposium on Wearable Computers, p.57, October 07-10, 2002
• 26: wearIT@work Project Homepage, http://www.wearitatwork.com/, 7 2007.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human factors

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Input devices and strategies (e.g., mouse, touchscreen)

General Terms:
Human Factors

Keywords:
interaction devices, interaction resource model, multimodal interaction, wearable computing