ShadowPlay: a generative model for nonverbal human-robot interaction

ABSTRACT

Humans rely on a finely tuned ability to recognize and adapt to socially relevant patterns in their everyday face-to-face interactions. This allows them to anticipate the actions of others, coordinate their behaviors, and create shared meaning to communicate. Social robots must likewise be able to recognize and perform relevant social patterns, including interactional synchrony, imitation, and particular sequences of behaviors. We use existing empirical work in the social sciences and observations of human interaction to develop nonverbal interactive capabilities for a robot in the context of shadow puppet play, where people interact through shadows of hands cast against a wall. We show how information theoretic quantities can be used to model interaction between humans and to generate interactive controllers for a robot. Finally, we evaluate the resulting model in an embodied human-robot interaction study. We show the benefit of modeling interaction as a joint process rather than modeling individual agents.

REFERENCES

• 1: Cynthia Breazeal, Designing Sociable Robots, MIT Press, Cambridge, MA, 2002
• 2: G. R. Semin, "Grounding communication: Synchrony, Social Psychology: Handbook of Basic Principles 2nd Edition, pp. 630--649, 2007.
• 3: K. Dautenhahn, B. Ogden, and T. Quick,"From embodied to socially embedded agents' Implications for interaction-aware robots,"Cognitive Systems Research, vol. 3, pp. 397--428, 2002.
• 4: L.W. Barsalou, C. Breazeal, and L. B. Smith, Cognition as coordinated non-cognition,"Cognitive Processing, vol. 8, no. 2, pp. 79--91, June 2007.
• 5: E. R. Smith and G. R. Semin, Socially situated cognition: Cognition in its social context, Advances in Experimental Social Psychology, vol. 36, pp. 53--117, 2004.
• 6: S. Sabanovic, M. P. Michalowski, and L. R. Caporael, Making friends: Building social robots through interdisciplinary collaboration, in Multidisciplinary Collaboration for Socially Assistive Robotics: Papers from the 2007 AAAI Spring Symposium, Technical Report SS-07-07. AAAI, March 2007, pp. 71--77.
• 7: W. S. Condon and L.W. Sander,"Synchrony demonstrated between movements of the neonate and adult speech,"Child Development, vol. 45, no. 2, pp. 456--462, 1974.
• 8: D. McNeill, Gesture & Thought. Chicago: Chicago University Press, 2005.
• 9: C. Crick, M. Munz, T. Nad, and B. Scassellati, Robotic drumming: Synchronization in social tasks, in IEEE ROMAN, 2006, pp. 97--102.
• 10: Marek P. Michalowski , Selma Sabanovic , Hideki Kozima, A dancing robot for rhythmic social interaction, Proceedings of the ACM/IEEE international conference on Human-robot interaction, March 10-12, 2007, Arlington, Virginia, USA [doi>10.1145/1228716.1228729]
• 11: H. Ogawa and T. Watanabe, Interrobot: speech-driven embodied interaction robot," Advanced Robotics, vol. 15, pp. 371--377, 2001.
• 12: H. Clark and S. A. Brennan, Perspectives on socially shared cognition. Academic Press, 1991, ch. Grounding in communication.
• 13: L. Barrett,"Too much monkey business,"in Proc. Grounding Sociality: Neurons, Minds, and Culture, 2007.
• 14: J. K. Burgoon, L. A. Stern, and L. Dillman, Interpersonal Adaptation: Dyadic Interaction Patterns. Cambridge: Cambridge University Press, 1995.
• 15: L. R. Caporael, The evolution of truly social cognition: The core configuration model,"Personality and Social Psychology Review, vol. 1, pp. 276--298, 1997.
• 16: E. Z. Tronick and J. F. Cohn, "Infant-mother face-to-face interaction: age and gender differences in coordination and the occurrence of miscoordination,"Child Development, vol. 60, no. 1, pp. 85--92, 1989.
• 17: T. L. Chartrand and J. A. Bargh,"The chameleon effect: The perception-behavior link and social interaction, Journal of Personality and Social Psychology, vol. 76, pp. 893--910, 1999.
• 18: P. Andry, P. Gaussier, S. Moga, J. P. Banquet, and J. Nadel,"Learning and communication via imitation: An autonomous robot perspective, IEEE Trans. Systems, Man, and Cybernetics, vol. 31, no. 5, pp. 431--442, 2000.
• 19: S. Penny, Embodied cultural agents: at the intersection of robotics, cognitive science, and interactive art, in AAAI Socially Intelligent Agents Symposium, 1997.
• 20: Mike Blow , Kerstin Dautenhahn , Andrew Appleby , Chrystopher L. Nehaniv , David Lee, The art of designing robot faces: dimensions for human-robot interaction, Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction, March 02-03, 2006, Salt Lake City, Utah, USA [doi>10.1145/1121241.1121301]
• 21: H. Kozima and C. Nakagawa, Social robots for children: Practice in communication-care, in Proc. IEEE Intl. Workshop on Advanced Motion Control, 2006, pp. 768--773.
• 22: F. J. Bernieri, J. S. Reznick, and R. Rosenthal, Synchrony, pseudosynchrony, and dissynchrony: Measuring the entrainment process in mother-infant interactions,"Journal of Personality and Social Psychology, vol. 54, no. 2, pp. 243--253, 1988.
• 23: Meisner and Sabanovic,"Shadow puppet interaction site,"May 2008, www.cs.rpi.edu/~meisne/interaction.
• 24: M.-K. Hu,"Visual pattern recognition by moment invariants, Information Theory, IEEE Transactions on, vol. 8, no. 2, pp. 179--187, 1962.
• 25: M. Cuturi and J.-P. Vert, A mutual information kernel for sequences, Proc. IEEE Intl. Joint Conf. on Neural Networks, vol. 3, pp. 1905--1910 vol.3, July 2004.
• 26: M. Seeger,"Covariance kernels from bayesian generative models, in NIPS, vol. 14, 2002, pp. 905--912.
• 27: Meisner, www.cs.rpi.edu/~meisne/interaction/wam.html.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.11 Distributed Artificial Intelligence
            Subjects: Coherence and coordination

General Terms:
Human Factors, Measurement

Keywords:
control architecture, gesture recognition, interaction synchrony, modeling social situations, nonverbal interaction