RoleNet: treat a movie as a small society

ABSTRACT

We present a brave new way to analyze movie content, from the perspectives of the relationships between roles rather than low-level audiovisual features. Interactions between roles in a movie resemble human behaviors in a society. Roles' actions lead the story and make viewers understand what directors want to present. In this paper, we introduce the idea of social network analysis to model the relationships of actors/actresses as a network, called RoleNet. Through analyzing this network, the proposed approach automatically determines the leading roles and the communities embedded in movies. We also describe an implementation framework to realize the proposed model. The experimental results show that the proposed methods can effectively capture social characteristics in movies. It's believed that this idea provides a different way to approach movie understanding.

REFERENCES

• 1: Rasheed, Z., Sheikh, Y., and Shah, M. On the use of computable features for film classification. IEEE Transactions on Circuits and Systems for Video Technology, vol. 15, no. 1, 2005, pp. 52--64.
• 2: Adams, B., Dorai, C., and Venkatesh, S. Toward automatic extraction and expression of expressive elements from motion pictures: tempo. IEEE Transactions on Multimedia, vol. 4, no. 4, 2002, pp. 472--481.
• 3: Vendrig, J. and Worring, M. Systematic evaluation of logical story unit segmentation. IEEE Transactions on Multimedia, vol. 4, no. 4, 2002, pp. 492--499.
• 4: Li, Y., Lee, S.-H., Yeh, C.-H., and Kuo, C.-C. J. Techniques for movie content analysis and skimming: tutorial and overview on video abstraction techniques. IEEE Signal Processing Magazine, vol. 23, no. 2, 2006, pp. 79--89.
• 5: Ba Tu Truong , Svetha Venkatesh, Video abstraction: A systematic review and classification, ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP), v.3 n.1, p.3-es, February 2007 [doi>10.1145/1198302.1198305]
• 6: Byunghee Jung , Taeyeong Kwak , Junehwa Song , Yoonjoon Lee, Narrative abstraction model for story-oriented video, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027720]
• 7: Alan F. Smeaton , Bart Lehane , Noel E. O'Connor , Conor Brady , Gary Craig, Automatically selecting shots for action movie trailers, Proceedings of the 8th ACM international workshop on Multimedia information retrieval, October 26-27, 2006, Santa Barbara, California, USA [doi>10.1145/1178677.1178709]
• 8: Hsuan-Wei Chen , Jin-Hau Kuo , Wei-Ta Chu , Ja-Ling Wu, Action movies segmentation and summarization based on tempo analysis, Proceedings of the 6th ACM SIGMM international workshop on Multimedia information retrieval, October 15-16, 2004, New York, NY, USA [doi>10.1145/1026711.1026752]
• 9: Wang, H.L. and Cheong, L.-F. Affective understanding in film. IEEE Transactions on Circuits and Systems for Video Technology, vol. 16, no. 6, 2006, pp. 689--704.
• 10: Hanjalic, A. and Xu, L.-Q. Affective video content representation and modeling. IEEE Transactions on Multimedia, vol 7, no. 1, 2005, pp. 143--154.
• 11: Scott, J. Social network analysis: a handbook. Newbury Park, 1991.
• 12: Guimera, R., Danon, L., Diaz-Guilera A., Giralt, F., and Arenas, A. Self-similar community structure in a network of human interactions. Physical Review, vol. 68, 065103(R), 2003.
• 13: Krause, A.E., Frank, K.A., Mason, D.M., Ulanowicz, R.E., and Taylor, W.W. Compartments revealed in food-web structure, Nature, vol. 426, 2003, pp. 282--285.
• 14: Yuri Boykov , Vladimir Kolmogorov, An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.26 n.9, p.1124-1137, September 2004 [doi>10.1109/TPAMI.2004.60]
• 15: Rasheed, Z., and Shah, M. Scene detection in Hollywood movies and TV shows. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, 2003, pp. 343--348.
• 16: Open Source Computer Vision Library, http://www.intel.com/technology/computing/opencv/
• 17: A. V. Nefian, An embedded HMM-based approach for face detection and recognition, Proceedings of the Acoustics, Speech, and Signal Processing, 1999. on 1999 IEEE International Conference, p.3553-3556, March 15-19, 1999 [doi>10.1109/ICASSP.1999.757610]
• 18: Zhu, X., Elmagarmid, A.K., Xue, X., Wu, L., and Catlin, A.C. InsightVideo: toward hierarchical video content organization for efficient browsing, summarization and retrieval. IEEE Transactions on Multimedia, vol. 7, no. 4, 2005, pp. 648--666.
• 19: H. J. Zhang , C. Y. Low , S. W. Smoliar , J. H. Wu, Video parsing, retrieval and browsing: an integrated and content-based solution, Proceedings of the third ACM international conference on Multimedia, p.15-24, November 05-09, 1995, San Francisco, California, United States [doi>10.1145/217279.215068]
• 20: Weng, C.-Y., Chu, W.-T., and Wu, J.-L. Movie analysis based on roles' social network. In Proceedings of IEEE International Conference on Multimedia & Expo, pp. 1403--1406, 2007.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Indexing methods

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.10 Vision and Scene Understanding
            Subjects: Modeling and recovery of physical attributes; Representations, data structures, and transforms

General Terms:
Algorithms, Human Factors, Management

Keywords:
RoleNet, movie analysis, social network analysis