Enabling always-available input with muscle-computer interfaces

ABSTRACT

Previous work has demonstrated the viability of applying offline analysis to interpret forearm electromyography (EMG) and classify finger gestures on a physical surface. We extend those results to bring us closer to using muscle-computer interfaces for always-available input in real-world applications. We leverage existing taxonomies of natural human grips to develop a gesture set covering interaction in free space even when hands are busy with other objects. We present a system that classifies these gestures in real-time and we introduce a bi-manual paradigm that enables use in interactive systems. We report experimental results demonstrating four-finger classification accuracies averaging 79% for pinching, 85% while holding a travel mug, and 88% when carrying a weighted bag. We further show generalizability across different arm postures and explore the tradeoffs of providing real-time visual feedback.

REFERENCES

• 1: Burges, C. 1998. A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 2, 121--167.
• 2: Costanza, E., Inverso, S.A., Allen, R., & Maes, P. 2007. Intimate interfaces in action: Assessing the usability and subtlety of EMG-based motionless gestures. CHI '07, 819--828.
• 3: Englehart, K. & Hudgins, B. 2003. A robust, real time control scheme for multifunction myoelectric control. IEEE Trans Biomedical Engineering, 50(7), 848--854.
• 4: Farry K., Walker I. & Baraniuk R. G. 1996. Myoelectric teleoperation of a complex robotic hand. Proc IEEE Int Conf Robot Autom, 775--788.
• 5: Greenberg, S. & Fitchett, C. 2001. Phidgets: easy development of physical interfaces through physical widgets. UIST '01, 209--218.
• 6: Ju, P., Kaelbling, L. P. & Singer, Y. 2000. State-based Classification of Finger Gestures from Electromyographic Signals. ICML '08, 439--446.
• 7: Kiguchi, K., Tanaka, T. & Fukuda, T. 2004. Neuro-fuzzy control of a robotic exoskeleton with EMG signals. IEEE Trans. on Fuzzy Systems, 12(4), 481--490.
• 8: Lakshmipathy, V., Schmandt, C., and Marmasse, N. 2003. TalkBack: a conversational answering machine. UIST '03.
• 9: Lanyi, X. & Adler, A. 2004. An improved method for muscle activation detection during gait. Canadian Conf. on Electrical and Computer Engineering, 357--360.
• 10: Lee, J.C. & Tan, D.S. 2006. Using a low-cost encephalograph for task classification in HCI research. UIST '06, 81--90.
• 11: Lyons, K., Skeels, C., Starner, T., Snoeck, C. M., Wong, B. A. & Ashbrook, D. 2004. Augmenting conversations using dual-purpose speech. UIST '04, 237--246.
• 12: MacKenzie, C. L. and Iberall, T. 1994. The Grasping Hand. Amsterdam: North-Holland, Elsevier Science.
• 13: Merletti, R., & Parker, P.A. 2004. Electromyography: Physiology, engineering, and noninvasive applications. John Wiley & Sons: Hoboken, New Jersey.
• 14: Naik, G.R., Kumar, D.K., Singh, V.P. & Palaniswami, M. 2006. Hand gestures for HCI using ICA of EMG. HCSNet Workshop on the Use of Vision in HCI, 67--72.
• 15: Peleg, D., Braiman, E., Yom-Tov, E. & Inbar G.F. 2002. Classification of Finger Activation for Use in a Robotic Prosthesis Arm. Trans Neural Syst Rehabil Eng, 10(4).
• 16: Platt, J. 1998. Sequential Minimal Optimization: A fast algorithm for training support vector machines. Microsoft Research Technical Report MSR-TR-98-14.
• 17: Raez, M.B.I., Hussain, M.S. & Mohd-Yasin, F. 2006. Techniques of EMG signal analysis: detection, processing, classification, and applications. Biological Procedures Online, 8, 11--35.
• 18: Saponas, T. S., Tan, D. S., Morris, D. & Balakrishnan, R. Demonstrating the feasibility of using forearm electromyography for muscle-computer interfaces. CHI '08, 515--524.
• 19: Sassaroli, A., Zheng, F., Hirshfield, L.M., Girouard, A., Solovey, E.T., Jacob, R.J.K. & Fantini, S. 2008. Discrimination of Mental Workload Levels in Human Subjects with Functional Near-Infrared Spectroscopy. J Innovative Optical Health Sciences, 1(2), 227--237.
• 20: Schlesinger, G. Der Adechanische Auflau der kunstlichen Glieder. 1919. Ersatzglieder und Arbeitshilfen, 1. Springer Verlag, Berlin.
• 21: Tenore, F., Ramos, A., Fahmy, A., Acharya, S., Etienne-Cummings, R. & Thakor, N. 2007. Towards the Control of Individual Fingers of a Prosthetic Hand Using Surface EMG Signals. IEEE EMBS.
• 22: Wang, G., Wang, Z., Chen, W. & Zhuang, J. 2006. Classification of Surface EMG signals using optimal wavelet packet method based on Davies-Bouldin criterion. Med Biol Eng Comput 44, 865--872.
• 23: Wheeler, K.R, Chang M.H. & Knuth K.H. 2006. Gesture-Based Control and EMG Decomposition. IEEE Trans. on Systems, Man, and Cybernetics, 36(4).
• 24: Wilson, A. 2005. PlayAnywhere: a compact interactive tabletop projection-vision system. UIST '05, 83--92.
• 25: Yatsenko, D., McDonnall D. & Guillory, S. 2007. Simultaneous, Proportional, Multi-axis Prosthesis Control using Multichannel Surface EMG. IEEE EMBS.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Input devices and strategies (e.g., mouse, touchscreen)

General Terms:
Design, Human Factors

Keywords:
electromyography (EMG), input, interaction, muscle-computer interface