Learning ontology for personalized video retrieval

ABSTRACT

This paper proposes a new method for using implicit user feedback from clickthrough data to provide personalized ranking of results in a video retrieval system. The annotation based search is complemented with a feature based ranking in our approach. The ranking algorithm uses belief revision in a Bayesian Network, which is derived from a multimedia ontology that captures the probabilistic association of a concept with expected video features. We have developed a content model for videos using discrete feature states to enable Bayesian reasoning and to alleviate on-line feature processing overheads. We propose a reinforcement learning algorithm for the parameters of the Bayesian Network with the implicit feedback obtained from the clickthrough data.

REFERENCES

• 1: Marin Ferecatu , Michel Crucianu , Nozha Boujemaa, Retrieval of difficult image classes using svd-based relevance feedback, Proceedings of the 6th ACM SIGMM international workshop on Multimedia information retrieval, October 15-16, 2004, New York, NY, USA [doi>10.1145/1026711.1026716]
• 2: N. Friedman and M. Goldszmidt. Sequential update of bayesian network structure. In Proceedings of the Thirteenth Conference on Uncertainty in Artificial Intelligence (UAI 97), 1997.
• 3: H. Ghosh. R-MAGIC: A cooperative agent based architecture for retrieval of multimedia documents distributed over heterogeneous repositories. PhD thesis, Department of Electrical Engineering, Indian Institute of Technology, Delhi, 2002.
• 4: H. Ghosh, S. Chaudhury, K. Kashyap, and B. Maiti. Ontology Specification and Integration for Multimedia Applications, chapter 9. Ontologies: A Handbook of Principles, Concepts and Applications in Information Systems. Springer, 2006.
• 5: G. Iyengar , P. Duygulu , S. Feng , P. Ircing , S. P. Khudanpur , D. Klakow , M. R. Krause , R. Manmatha , H. J. Nock , D. Petkova , B. Pytlik , P. Virga, Joint visual-text modeling for automatic retrieval of multimedia documents, Proceedings of the 13th annual ACM international conference on Multimedia, November 06-11, 2005, Hilton, Singapore [doi>10.1145/1101149.1101154]
• 6: Thorsten Joachims, Optimizing search engines using clickthrough data, Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, July 23-26, 2002, Edmonton, Alberta, Canada [doi>10.1145/775047.775067]
• 7: T. Karthik, S. Chaudhury, and H. Ghosh. Specifying spatio-temporal relations in multimedia ontologies. In Proceedings of International Conference of Pattern Recognition and Machine Intelligence (PReMI'05), 2005.
• 8: A. Mallik, A. Jain, M. Matela, S. Chaudhury, and S. Bhattacharya. Multimodal retrieval strategy for video collections. IJCAI-2007 Workshop on Multimodal Information Retrieval, 2007.
• 9: B.S. Manjunath , Phillipe Salembier , Thomas Sikora, Introduction to MPEG-7: Multimedia Content Description Interface, John Wiley & Sons, Inc., New York, NY, 2002
• 10: P. Muneesawang , L. Guan, Automatic relevance feedback for video retrieval, Proceedings of the 2003 International Conference on Multimedia and Expo, p.373-376, July 06-09, 2003
• 11: M. R. Naphade, S. Basu, J. R. Smith, C.-Y. Lin, and B. Tseng. A statistical modeling approach to content based video retrieval. In 16th International Conference on Pattern Recognition (ICPR'02), pages 953--956, October 2002.
• 12: P. Norman. Putting iterative proportional fitting on the researcher's desk. Research Working Paper 99/3, School of Geography, University of Leeds(UK), 1999.
• 13: Takashi Onoda , Hiroshi Murata , Seiji Yamada, Support Vector Machines Based Active Learning for the Relevance Feedback Document Retrieval, Proceedings of the 2006 IEEE/WIC/ACM international conference on Web Intelligence and Intelligent Agent Technology, p.389-392, December 18-22, 2006 [doi>10.1109/WI-IATW.2006.125]
• 14: P. Quelhas and J.M. Odobez. Natural scene image modeling using color and texture visterms. In Proceedings International Conference on Image and Video Retrieval (CIVR). ACM Press, July 2006.
• 15: Filip Radlinski , Thorsten Joachims, Query chains: learning to rank from implicit feedback, Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining, August 21-24, 2005, Chicago, Illinois, USA [doi>10.1145/1081870.1081899]
• 16: R. Wang, M. Naphade, and T. Huang. Video retrieval and relevance feedback in the context of a post-integration model. In Proceedings of IEEE Multimedia Signal Processing (MMSP2001), 2001.
• 17: Wikipedia. Arrow's Impossibility Theorem. http://en.wikipedia.org/wiki/Arrow's_impossibility_theorem.
• 18: Wensi Xi , Jesper Lind , Eric Brill, Learning effective ranking functions for newsgroup search, Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, July 25-29, 2004, Sheffield, United Kingdom [doi>10.1145/1008992.1009060]
• 19: Gui-Rong Xue , Hua-Jun Zeng , Zheng Chen , Yong Yu , Wei-Ying Ma , WenSi Xi , WeiGuo Fan, Optimizing web search using web click-through data, Proceedings of the thirteenth ACM international conference on Information and knowledge management, November 08-13, 2004, Washington, D.C., USA [doi>10.1145/1031171.1031192]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning
  I.5 PATTERN RECOGNITION
      I.5.4 Applications

General Terms:
Algorithms, Experimentation, Human Factors

Keywords:
bayesian network, content modeling, reinforcement learning, video retrieval