Evaluation of an on-line adaptive gesture interface with command prediction

ABSTRACT

We present an evaluation of a hybrid gesture interface framework that combines on-line adaptive gesture recognition with a command predictor. Machine learning techniques enable on-line adaptation to differences in users' input patterns when making gestures, and exploit regularities in command sequences to improve recognition performance. A prototype using 2D single-stroke gestures was implemented with a minimally intrusive user interface for on-line re-training. Results of a controlled user experiment show that the hybrid adaptive system significantly improved overall gesture recognition performance, and reduced users' need to practice making the gestures before achieving good results.

REFERENCES

• 1: Gary Boone, Concept features in Re:Agent, an intelligent Email agent, Proceedings of the second international conference on Autonomous agents, p.141-148, May 10-13, 1998, Minneapolis, Minnesota, United States [doi>10.1145/280765.280791]
• 2: Brusilovsky, P. & Pesin, L. (1998). Adaptive navigation support in educational hypermedia: An evaluation of the ISIS-Tutor. Journal of Computing and Information Technology, 6(1). p.27--38.
• 3: Xiang Cao , Ravin Balakrishnan, VisionWand: interaction techniques for large displays using a passive wand tracked in 3D, Proceedings of the 16th annual ACM symposium on User interface software and technology, p.173-182, November 02-05, 2003, Vancouver, Canada [doi>10.1145/964696.964716]
• 4: Cleary, J. & Witten, I. (1984). Data compression using adaptive coding and partial string matching. IEEE Trans. on Comm, 32(4). p. 396--402.
• 5: Scott D. Connell , Anil K. Jain, Writer Adaptation for Online Handwriting Recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.3, p.329-346, March 2002 [doi>10.1109/34.990135]
• 6: Fu, H., Chang, H., Yeong, Y., & Pao, H. (2000). User adaptive handwriting recognition by self-growing probabilistic decision-based neural networks. IEEE Trans. on Neural Networks, 11(6).
• 7: Hermens, L. & Schlimmer, J. (1993). A machine-learning apprentice for the completion of repetitive forms. IEEE Conf on AI Applications. p. 164--170.
• 8: Ken Hinckley, Synchronous gestures for multiple persons and computers, Proceedings of the 16th annual ACM symposium on User interface software and technology, p.149-158, November 02-05, 2003, Vancouver, Canada [doi>10.1145/964696.964713]
• 9: Hinkle, D. & Toomey, C. (1994). CLAVIER: Applying case-based reasoning on to composit part fabrication. 6th Innovative App. of AI Conf. p. 55--62.
• 10: Lang, K. (1995). News Weeder: Learning to filter news. Intl. Conf. on Machine Learning. p. 331--339.
• 11: Pat Langley, User modeling in adaptive interfaces, Proceedings of the seventh international conference on User modeling, p.357-370, June 1999, Banff, Canada
• 12: Jennifer Mankoff , Scott E. Hudson , Gregory D. Abowd, Interaction techniques for ambiguity resolution in recognition-based interfaces, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.11-20, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354407]
• 13: Joanna McGrenere , Ronald M. Baecker , Kellogg S. Booth, An evaluation of a multiple interface design solution for bloated software, Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/503376.503406]
• 14: Takahiro Nishi , Yoichi Sato , Hideki Koike, Interactive object registration and recognition for augmented desk interface, CHI '01 extended abstracts on Human factors in computing systems, March 31-April 05, 2001, Seattle, Washington [doi>10.1145/634067.634286]
• 15: Pazzani, M., Muramatsu, J., and Billsus, D. (1996). Syskill & Webert: Identifying interesting web sites. Natl. Conference on Artificial Intelligence. p. 54--61.
• 16: Rabiner, L. R. (1989). A tutorial on Hidden Markov Models and selected applications in speech recognition. Proceedings of the IEEE, 77(2). p.257--285.
• 17: Dean Rubine, Specifying gestures by example, Proceedings of the 18th annual conference on Computer graphics and interactive techniques, p.329-337, July 1991
• 18: Jakub Segen , Senthil Kumar, Gesture VR: vision-based 3D hand interace for spatial interaction, Proceedings of the sixth ACM international conference on Multimedia, p.455-464, September 13-16, 1998, Bristol, United Kingdom [doi>10.1145/290747.290822]
• 19: Jakub Segen , Senthil Kumar, Look ma, no mouse!, Communications of the ACM, v.43 n.7, p.102-109, July 2000 [doi>10.1145/341852.341869]
• 20: Vuori, V. (2002). Adaptive Methods for On-Line Recognition of Isolated Handwritten Characters. Ph.D. Thesis. Helsinki University of Technology.
• 21: David J. Ward , Alan F. Blackwell , David J. C. MacKay, Dasher—a data entry interface using continuous gestures and language models, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.129-137, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354427]
• 22: Weber, G. & Specht, M. (1997). User modeling and adaptive navigation support in WWW-based tutoring systems. Intl. Conf. User Modeling. p. 289--300.
• 23: Andrew David Wilson , Aaron F. Bobick , Bruce M. Blumberg, Adaptive models for the recognition of human gesture, 2000
• 24: Andrew D. Wilson , Aaron F. Bobick, Real-Time Online Adaptive Gesture Recognition, Proceedings of the International Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems, p.111, September 26-27, 1999
• 25: Wilson, A. & Pham, H. (2003). Pointing in intelligent environments with the World Cursor. Interact.
• 26: Andrew Wilson , Steven Shafer, XWand: UI for intelligent spaces, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642706]
• 27: Terry Winograd , François Guimbretière, Visual instruments for an interactive mural, CHI '99 extended abstracts on Human factors in computing systems, May 15-20, 1999, Pittsburgh, Pennsylvania [doi>10.1145/632716.632861]
• 28: Mike Wu , Ravin Balakrishnan, Multi-finger and whole hand gestural interaction techniques for multi-user tabletop displays, Proceedings of the 16th annual ACM symposium on User interface software and technology, p.193-202, November 02-05, 2003, Vancouver, Canada [doi>10.1145/964696.964718]

INDEX TERMS

Keywords:
adaptive interfaces, gestures, user studies