Video agent: interactive autonomous agents generated from real-world creatures

ABSTRACT

We present a novel approach for interactive multimedia content creation that establishes an interactive environment in cyberspace in which users interact with autonomous agents generated from video images of real-world creatures. Each agent has autonomy, personality traits, and behaviors that reflect the results of various interactions determined by an emotional model with fuzzy logic. After an agent's behavior is determined, a sequence of video images that best match the determined behavior is retrieved from the database in which a variety of video image sequences of the real creature's behaviors are stored. The retrieved images are successively displayed on the cyberspace to make it responsive. Thus the autonomous agent behaves continuously. In addition, an explicit sketch-based method directly initiate the reactive behavior of the agent without involving the emotional process. This paper describes the algorithm that establishes such an interactive system. First, an image processing algorithm to generate a video database is described. Then the process of behavior generation using emotional models and sketch-based instruction are introduced. Finally, two application examples are demonstrated: video agents with humans and goldfish.

REFERENCES

• 1: Kazuhiro Asai , Takashi Nishimoto , Yoshinori Hirano , Emiko Hama , Yoshifumi Kitamura , Fumio Kishino, Video agents, ACM SIGGRAPH 2007 emerging technologies, August 05-09, 2007, San Diego, California [doi>10.1145/1278280.1278305]
• 2: Balaguer, J., and Gobbetti, E. 1995. Sketching 3D animations. Computer Graphics Forum 14, 3, 241--258.
• 3: Bruce Blumberg , Marc Downie , Yuri Ivanov , Matt Berlin , Michael Patrick Johnson , Bill Tomlinson, Integrated learning for interactive synthetic characters, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 4: Chetverikov, D., and Szabó, Z. 1999. A simple and efficient algorithm for detection of high curvature points in planar curves. In Workshop of Austrian Pattern Recognition Group, 175--184.
• 5: Costa, P. T., and McCrae, R. R. 1985. The NEO Personality Inventory. Psychological Assessment Resources.
• 6: Christina de Juan , Bobby Bodenheimer, Cartoon textures, Proceedings of the 2004 ACM SIGGRAPH/Eurographics symposium on Computer animation, August 27-29, 2004, Grenoble, France [doi>10.1145/1028523.1028559]
• 7: Stéphane Donikian, HPTS: a behaviour modelling language for autonomous agents, Proceedings of the fifth international conference on Autonomous agents, p.401-408, May 2001, Montreal, Quebec, Canada [doi>10.1145/375735.376382]
• 8: Ekman, P., and Davidson, R. J. 1994. The nature of emotion: fundamental questions. Oxford University Press.
• 9: Clark Davidson Elliott, The affective reasoner: a process model of emotions in a multi-agent system, Northwestern University, Evanston, IL, 1992
• 10: Frijda, N. 1986. The emotions. Cambridge University Press.
• 11: Stacy Kombis, Virtual FishTank, ACM SIGGRAPH 98 Conference abstracts and applications, p.116, July 19-24, 1998, Orlando, Florida, United States [doi>10.1145/280953.281310]
• 12: Lucas Kovar , Michael Gleicher , Frédéric Pighin, Motion graphs, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 13: Kruskal, J. B., and Seery, J. B. 1980. Designing network diagrams. In Proc. of General Conference on Social Graphics, 22--50.
• 14: I-Jong Lin , S. Y. Kung, Video Object Extraction and Representation: Theory and Applications, Kluwer Academic Publishers, Norwell, MA, 2000
• 15: A. Bryan Loyall , W. Scott Neal Reilly , Joseph Bates , Peter Weyhrauch, System for authoring highly interactive, personality-rich interactive characters, Proceedings of the 2004 ACM SIGGRAPH/Eurographics symposium on Computer animation, August 27-29, 2004, Grenoble, France [doi>10.1145/1028523.1028532]
• 16: Marvin Minsky, The society of mind, Simon & Schuster, Inc., New York, NY, 1986
• 17: Jovan Popović , Steven M. Seitz , Michael Erdmann, Motion sketching for control of rigid-body simulations, ACM Transactions on Graphics (TOG), v.22 n.4, p.1034-1054, October 2003 [doi>10.1145/944020.944025]
• 18: Radica, 2005. Cube world. http://www.radicauk.com/.
• 19: Craig W. Reynolds, Flocks, herds and schools: A distributed behavioral model, Proceedings of the 14th annual conference on Computer graphics and interactive techniques, p.25-34, August 1987
• 20: Ronald, A., Fujita, M., Takagi, T., and Hasegawa, R. 2003. An ethological and emotional basis for human-robot interaction. Robotics and Autonomous Systems 42, 191--201.
• 21: Arno Schödl , Irfan A. Essa, Controlled animation of video sprites, Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation, July 21-22, 2002, San Antonio, Texas [doi>10.1145/545261.545281]
• 22: Arno Schödl , Richard Szeliski , David H. Salesin , Irfan Essa, Video textures, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.489-498, July 2000 [doi>10.1145/344779.345012]
• 23: Sloman, A. 1997. Designing human-like minds. In Proc. of European Conference on Artificial Life.
• 24: J. Starck , G. Miller , A. Hilton, Video-based character animation, Proceedings of the 2005 ACM SIGGRAPH/Eurographics symposium on Computer animation, July 29-31, 2005, Los Angeles, California [doi>10.1145/1073368.1073375]
• 25: Wen-Poh Su , Binh Pham , Aster Wardhani, High-level control posture of story characters based on personality and emotion, Proceedings of the second Australasian conference on Interactive entertainment, p.179-186, November 23-25, 2005, Sydney, Australia
• 26: Sun, J., Zhang, W., Tang, X., and Shum, H.-Y. 2006. Background cut. In Proc. of European Conference on Computer Vision, 628--641.
• 27: Xiaoyuan Tu , Demetri Terzopoulos, Artificial fishes: physics, locomotion, perception, behavior, Proceedings of the 21st annual conference on Computer graphics and interactive techniques, p.43-50, July 1994 [doi>10.1145/192161.192170]
• 28: Matthew Thorne , David Burke , Michiel van de Panne, Motion doodles: an interface for sketching character motion, ACM Transactions on Graphics (TOG), v.23 n.3, August 2004
• 29: Bill Tomlinson , Marc Downie , Matt Berlin , Jesse Gray , Derek Lyons , Jennie Cochran , Bruce Blumberg, Leashing the AlphaWolves: mixing user direction with autonomous emotion in a pack of semi-autonomous virtual characters, Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation, July 21-22, 2002, San Antonio, Texas [doi>10.1145/545261.545263]
• 30: Hirohide Ushida , Yuji Hirayama , Hiroshi Nakajima, Emotion model for life-like agent and its evaluation, Proceedings of the fifteenth national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence, p.62-69, July 1998, Madison, Wisconsin, United States
• 31: Velasquez, J. D. 1997. Modeling emotions and other motivations in synthetic agents. In Proc. of Conference on Artificial Intelligence (AAAI), 10--15.
• 32: Jue Wang , Pravin Bhat , R. Alex Colburn , Maneesh Agrawala , Michael F. Cohen, Interactive video cutout, ACM SIGGRAPH 2005 Papers, July 31-August 04, 2005, Los Angeles, California

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.4 Graphics Utilities
          Subjects: Virtual device interfaces

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Artificial, augmented, and virtual realities

Keywords:
characters, computer animation, fuzzy logic, image processing, interactive multimedia content, video database