Motion overview of human actions

ABSTRACT

During the last decade, motion capture data has emerged and gained a leading role in animations, games and 3D environments. Many of these applications require the creation of expressive overview video clips capturing the human motion, however sufficient attention has not been given to this problem. In this paper, we present a technique that generates an overview video based on the analysis of motion capture data. Our method is targeted for applications of 3D character based animations, automating, for example, the action summary and gameplay overview in simulations and computer games. We base our method on quantum annealing optimization with an objective function that respects the analysis of the character motion and the camera movement constraints. It automatically generates a smooth camera control path, splitting it to several shots if required. To evaluate our method, we introduce a novel camera placement metric which is evaluated against previous work and conduct a user study comparing our results with the various systems.

REFERENCES

• 1: Apolloni, B., Carvalho, C., and de Falco, D. 1989. Quantum stochastic optimization. Stochastic Processes and their Applications 33, 2 (Dec), 233--244.
• 2: Arijon, D. 1976. Grammar of the film language. Silman-James Press.
• 3: Jackie Assa , Yaron Caspi , Daniel Cohen-Or, Action synopsis: pose selection and illustration, ACM SIGGRAPH 2005 Papers, July 31-August 04, 2005, Los Angeles, California
• 4: Bares, W. H., and Lester, J. C. 1999. Intelligent multi-shot 3d visualization interfaces. Knowl.-Based Syst. 12, 8, 403--412.
• 5: Bares, W. H., Thainimit, S., and McDermott, S. 2000. A model for constraint-based camera planning. In Smart Graphics, Papers from the 2000 AAAI Spring Symposium, AAAI Press, vol. 4, 84--91.
• 6: Alan H. Barr , Bena Currin , Steven Gabriel , John F. Hughes, Smooth interpolation of orientations with angular velocity constraints using quaternions, Proceedings of the 19th annual conference on Computer graphics and interactive techniques, p.313-320, July 1992
• 7: Frédéric Benhamou , Frédéric Goualard , Éric Languénou, , Marc Christie, Interval constraint solving for camera control and motion planning, ACM Transactions on Computational Logic (TOCL), v.5 n.4, p.732-767, October 2004 [doi>10.1145/1024922.1024927]
• 8: Blanz, V., Tarr, M. J., and Bülthoff, H. H. 1999. What object attributes determine canonical views. Perception 28, 5, 575--600.
• 9: Christianson, D. B., Anderson, S. E., wei He, L., Salesin, D. H., Weld, D. S., and Cohen, M. F. 1996. Declarative camera control for automatic cinematography. In AAAI/IAAI, Vol. 1, 148--155.
• 10: Christie, M., and Olivier, P. 2006. Camera control in computer graphics. In Eurographics 2006 Star Report, 89--113.
• 11: Christie, M., Machap, R., Normand, J.-M., Olivier, P., and Pickering, J. 2005. Virtual camera planning: A survey. In Smart Graphics, 40--52.
• 12: Daniel DeMenthon , Vikrant Kobla , David Doermann, Video summarization by curve simplification, Proceedings of the sixth ACM international conference on Multimedia, p.211-218, September 13-16, 1998, Bristol, United Kingdom [doi>10.1145/290747.290773]
• 13: Drucker, S. M., and Zeltzer, D. 1994. Intelligent camera control in a virtual environment. In Proceedings of Graphics Interface '94, 190--199.
• 14: Michael Gleicher , Andrew Witkin, Through-the-lens camera control, Proceedings of the 19th annual conference on Computer graphics and interactive techniques, p.331-340, July 1992
• 15: Gómez, F., Hurtado, F., Sellares, J. A., and Toussaint, G. T. 2001. Nice perspective projections. Journal of Visual Communication and Image Representation 12, 4, 387--400.
• 16: Bruce Gooch , Erik Reinhard , Chris Moulding , Peter Shirley, Artistic Composition for Image Creation, Proceedings of the 12th Eurographics Workshop on Rendering Techniques, p.83-88, June 25-27, 2001
• 17: Halper, N., and Olivier, P. 2000. Camplan: A camera planning agent. In AAAI 2000 Spring Symposium on Smart Graphics, AAAI Press, 92--100.
• 18: Halper, N., Helbing, R., and Strothotte, T. 2001. A camera engine for computer games: Managing the trade-off between constraint satisfaction and frame coherence. In EG 2001 Proceedings, Blackwell Publishing, vol. 20(3), 174--183.
• 19: Li-wei He , Michael F. Cohen , David H. Salesin, The virtual cinematographer: a paradigm for automatic real-time camera control and directing, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.217-224, August 1996 [doi>10.1145/237170.237259]
• 20: Jardillier, F., and Languénou, E. 1998. Screen-space constraints for camera movements: the virtual cameraman. Computer Graphics Forum 17, 3, 175--186. ISSN 1067--7055.
• 21: Tomihisa Kamada , Satoru Kawai, A simple method for computing general position in displaying three-dimensional objects, Computer Vision, Graphics, and Image Processing, v.41 n.1, p.43-56, Jan., 1988 [doi>10.1016/0734-189X(88)90116-8]
• 22: Katz, S. D. 1991. Film Directing Shot by Shot: Visualizing from Concept to Screen. Michael Wiese Productions.
• 23: Ji-Yong Kwon , In-Kwon Lee, Determination of camera parameters for character motions using motion area, The Visual Computer: International Journal of Computer Graphics, v.24 n.7, p.475-483, July 2008 [doi>10.1007/s00371-008-0228-x]
• 24: Ivan Laptev , Tony Lindeberg, Space-time Interest Points, Proceedings of the Ninth IEEE International Conference on Computer Vision, p.432, October 13-16, 2003
• 25: Jehee Lee , Jinxiang Chai , Paul S. A. Reitsma , Jessica K. Hodgins , Nancy S. Pollard, Interactive control of avatars animated with human motion data, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 26: Chang Ha Lee , Amitabh Varshney , David W. Jacobs, Mesh saliency, ACM SIGGRAPH 2005 Papers, July 31-August 04, 2005, Los Angeles, California
• 27: Lin, T.-C., Shih, Z.-C., and Tsai, Y.-T. 2004. Cinematic camera control in 3d computer games. In WSCG, 289--296.
• 28: Mascelli, J. V. 1965. The Five C's of Cinematography: Motion Picture Filming Techniques. Cine/Graphic Publications.
• 29: McCabe, H., and Kneafsey, J. 2006. A virtual cinematography system for first person shooter games. In Proceedings of International Digital Games Conference, 25--35.
• 30: Page, D. L., Koschan, A., Sukumar, S. R., Roui-Abidi, B., and Abidi, M. A. 2003. Shape analysis algorithm based on information theory. In ICIP (1), 229--232.
• 31: Palmer, S., Rosch, E., and Chase, P. 1981. Canonical perspective and the perception of objects. Attention and Performance IX, 135--151.
• 32: Min Je Park , Sung Yong Shin, Example-based motion cloning: Research Articles, Computer Animation and Virtual Worlds, v.15 n.3-4, p.245-257, July 2004 [doi>10.1002/cav.v15:3/4]
• 33: Pickering, J. H. 2002. Intelligent Camera Planning for Computer Graphics. PhD thesis, University of York.
• 34: Polonsky, O., Patanè, G., Biasotti, S., Gotsman, C., and Spagnuolo, M. 2005. What's in an image: Towards the computation of the "best" view of an object. The Visual Computer 21, 8--10, 840--847.
• 35: Ken Shoemake, Animating rotation with quaternion curves, Proceedings of the 12th annual conference on Computer graphics and interactive techniques, p.245-254, July 1985
• 36: Dmitry Sokolov , Dimitri Plemenos, Virtual world explorations by using topological and semantic knowledge, The Visual Computer: International Journal of Computer Graphics, v.24 n.3, p.173-185, January 2008 [doi>10.1007/s00371-007-0182-z]
• 37: Vázquez, P.-P., Feixas, M., Sbert, M., and Heidrich, W. 2003. Automatic view selection using viewpoint entropy and its application to image-based modelling. Computer Graphics Forum 22, 4, 689--700.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.10 Vision and Scene Understanding
            Subjects: Modeling and recovery of physical attributes

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.10 Vision and Scene Understanding
            Subjects: Motion
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.1 Digitization and Image Capture

General Terms:
Design, Performance

Keywords:
animation, animation summary, camera, mocap, salient action, viewpoint selection