Retrieval based interactive cartoon synthesis via unsupervised bi-distance metric learning

ABSTRACT

Cartoons play important roles in many areas, but it requires a lot of labor to produce new cartoon clips. In this paper, we propose a gesture recognition method for cartoon character images with two applications, namely content-based cartoon image retrieval and cartoon clip synthesis. We first define Edge Features (EF) and Motion Direction Features (MDF) for cartoon character images. The features are classified into two different groups, namely intra-features and inter-features. An Unsupervised Bi-Distance Metric Learning (UBDML) algorithm is proposed to recognize the gestures of cartoon character images. Different from the previous research efforts on distance metric learning, UBDML learns the optimal distance metric from the heterogeneous distance metrics derived from intra-features and inter-features. Content-based cartoon character image retrieval and cartoon clip synthesis can be carried out based on the distance metric learned by UBDML. Experiments show that the cartoon character image retrieval has a high precision and that the cartoon clip synthesis can be carried out efficiently.

REFERENCES

• 1: D.W. Liang, Y. Liu, Q.M. Huang, G.Y. Zhu, S.Q. Jiang, Z.B. Zhang and W. Gao. "Video2Cartoon: Generating 3D Cartoon from Broad-cast Soccer Video". In Proceedings of ACM Multimedia, 2005.
• 2: A. Verma, L.V. Subramaniam, N. Rajput, C. Neti and T.A. Faruquie. "Animating expressive faces across languages". IEEE Transactions on Multimedia, 6, 791 -- 800, 2004.
• 3: H. Chen, N.N. Zheng, L. Liang, Y. Li, Y.Q. Xu and H.Y. Shum. "PicToon: A Personalized Image-based Cartoon System". In Proceedings of ACM Multimedia, 2002.
• 4: Y. Li, F. Yu, Y. Q. Xu, E. Chang and H.Y. Shum. "Speech-Driven Cartoon Animation with Emotions". In Proceedings of ACM International Multimedia, 2001.
• 5: C.D. Juan and B. Bodenheimer. "Cartoon Textures". In Proceedings of. Symp. Computer Animation, 267--276, 2004.
• 6: Y. Zhuang, J. Yu, J. Xiao and C. Chen. "Perspective-aware Cartoon Clips Synthesis". Computer Animation and Virtual Worlds, 19(3), 355--364, 2008.
• 7: A. Kort. "Computer Aided Inbetweening". In Proceedings of International Symp. Non-photorealistic Animation and Rendering, 2002.
• 8: J.D. Fekete, E. Bizouarn, E. Cournarie, T. Galas and F. Taillefer. "Tic-TacToon: A Paperless System for Professional 2D Animation". In Proceedings of Computer Graphics and Interactive Techniques, 1995.
• 9: A. Bissacco, M.H. Yang, and S. Soatto. "Fast Human Pose Estimation using Appearance and Motion via Multi-Dimensional Boosting Regression". In Proceedings of CVPR, 2007.
• 10: T. Ding. "A robust identification approach to gait recognition". In Proceedings of CVPR, 2008.
• 11: K. Weinberger, J. Blitzer and L. Saul. "Distance metric learning for large margin nearest neighbor classification". In Proceedings of NIPS 2006.
• 12: Z. Zhang, J. Kwok, and D. Yeung. "Parametric distance metric learning with label information". In Proceedings of IJCAI, 2003.
• 13: M. Bilenko, S. Basu and R. J. Mooney. "Integrating constraints and metric learning in semi-supervised clustering". In Proceedings of ICML, 2004.
• 14: E. Xing, A. Ng, M. Jordan, and S. Russell. "Distance metric learning with application to clustering with side-information." In Proceedings of NIPS, 2003.
• 15: F. Nie, S. Xiang and C. Zhang. "Neighborhood minmax projections". In Proceedings of IJCAI, 2007.
• 16: L. Yang and R. Jin. "Distance metric learning: A comprehensive survey" Technical report, Michigan State University.
• 17: J.B. Tenenbaum, V.D. Silva, and J.C. Langford. "A global geometric framework for nonlinear dimensionality reduction". Science, 290, 2319--2323, 2000.
• 18: S. T. Roweis and L. K. Saul. "Nonlinear dimensionality reduction by locally linear embedding". Science, 290, 2323--2326, 2000.
• 19: K. Fukunaga. "Introduction to Statistical Pattern Classification". Academic Press.
• 20: I. Biederman and J. Gu, "Surface versus edge-based determinants of visual recognition". Cognitive Psychol, 20 38--64, 1988.
• 21: H. Hotelling. "Analysis of a Complex of Statistical Variables into Principal Components". Journal of Educational Psychology 24, 417--441, 1933.
• 22: D. Marr and E.C. Hildreth. "Theory of edge detection". In Proceedings of Royal Society, 1980.
• 23: H. Liu, T. Hong, M. Herman, T. Camus, and R. Chellappa. Accuracy vs. efficiency trade-offs in optical flow algorithms. CVIU, 72(3):271--286, 1998.
• 24: J. B. Kruskal, and M. Wish. "Multidimensional Scaling". Sage Publications. Beverly Hills. CA. 1977.
• 25: HA. SchödlH, HR. SzeliskiH, HD. H. SalesinH and HI. EssaH. "Video Textures". In Proceedings of SIGGRAPH, 2000.
• 26: Y. Li, T. Wang and H.Y. Shum. "Motion texture: a two-level statistical model for character motion synthesis". In Proceedings of SIGGRAPH, 2002.
• 27: X. He and P. Niyogi. "Locality Preserving Projections". In: Proceedings of NIPS, 2004.
• 28: D. Hoiem, A. Efros and M. Hebert. "Putting objects in perspective". International Journal of Computer Vision, 80(1): 3--15, 2008.
• 29: X.C. He and N.C. Yung. "Curvature Scale Space Corner Detector with Adaptive Threshold and Dynamic Region of Support". In Proceedings of ICPR, 2004.
• 30: J. Luo, J. Yu, D. Joshi and W. Hao, "Event recognition: viewing the world with a third eye". In Proceedings of ACM Multimedia, 2008.
• 31: S. Yan, D. Xu, B. Zhang, H. Zhang, Q. Yang and S. Lin, "Graph Embedding and Extensions: A General Framework for Dimensionality Reduction," IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(1):40--51, 2007.
• 32: T. Zhang, D. Tao, and J. Yang, "Discriminative Locality Alignment," In Proceedings of ECCV, 2008.
• 33: T. Zhang, D. Tao, X. Li, and J. Yang, "A Unifying Framework for Spectral Analysis based Dimensionality Reduction," In Proceedings of IJCNN, 2008.
• 34: D. Xu and S. Yan, "Semi-Supervised Bilinear Subspace Learning", IEEE Transactions on Image Processing, 18(7): 1671--1676, 2009.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Video (e.g., tape, disk, DVI)

General Terms:
Algorithms, Experimentation

Keywords:
cartoon clip synthesis, cartoon image, image retrieval