Building a visual ontology for video retrieval

ABSTRACT

To ensure access to growing video collections, annotation is becoming more and more important using background knowledge in the form of ontologies or thesauri is a way to facilitate annotation in a broad domain. Current ontologies are not suitable for (semi-) automatic annotation of visual resources as they contain little visual information about the concepts they describe. We investigate how an ontology that does contain visual information can facilitate annotation in a broad domain and identify requirements that a visual ontology has to meet. Based on these requirements, we create a visual ontology out of two existing knowledge corpora (WordNet and MPEG-7) by creating links between visual and general concepts. We test performance of the ontology on 40 shots of news video, and discuss the added value of each visual property.

REFERENCES

• 1: Grigoris Antoniou , Frank vanHarmelen, A Semantic Web Primer, MIT Press, Cambridge, MA, 2004
• 2: C. Fellbaum, editor. WordNet: An Electronic Lexical Database. Bradford Books, 1998.
• 3: Jan-Mark Geusebroek , Arnold W. M. Smeulders, A Six-Stimulus Theory for Stochastic Texture, International Journal of Computer Vision, v.62 n.1-2, p.7-16, April-May 2005 [doi>10.1007/s11263-005-4632-7]
• 4: S. Handschuh and S. Staab. Annotation for the Semantic Web, volume 96 of Frontiers in Artificial Intelligence and Applications. IOS Press, 2003.
• 5: A.G. Hauptmann. Towards a large scale concept ontology for broadcast video. In Proc. of the 3rd int conf on Image and Video Retrieval (CIVR'04), 2004.
• 6: M.A. Hoang, J. Geusebroek, and A.W.M. Smeulders. Color texture measurement and segmentation. In Proc. of Texture Analysis and Synthesis, 2002.
• 7: L. Hollink, A.Th. Schreiber, J. Wielemaker, and B. Wielinga. Semantic annotation of image collections. In Proc. of the Semannot Workshop, USA, 2003.
• 8: A. Hoogs, J. Rittscher, G. Stein, and J. Schmiederer. Video content annotation using visual analysis and a large semantic knowledgebase. In Proc. of IEEE Computer Vision and Pattern Recognition, 2003.
• 9: J. Hunter. Adding multimedia to the semantic web - building an mpeg-7 ontology. In International Semantic Web Working Symposium (SWWS), 2001.
• 10: E. Hyvonen, S. Saarela, K. Viljanen, E. Mäkelä, A. Valo, M. Salminen, S. Kettula, and M. Junnila. A cultural community portal for publishing museum collections on the semantic web. In Proc. of the ECAI Workshop on Appl. of Semantic Web Technologies to Web, 2004.
• 11: E. Lee. Building interoperability for united kingdom environment information resources. In Proc. of the European Conference on Digital Libraries, 2004.
• 12: J.M. Martinez. Overview of the mpeg-7 standard. Technical Report 5.0, ISO/IEC, Singapore, 2001.
• 13: José M. Martínez , César González , Oscar Fernández , Clara García , Jaime de Ramón, Towards universal access to content using MPEG-7, Proceedings of the tenth ACM international conference on Multimedia, December 01-06, 2002, Juan-les-Pins, France [doi>10.1145/641007.641044]
• 14: V. Mezaris, I. Kompatsiaris, and M.G. Strintzis. Region-based image retrieval using an object ontology and relevance feedback. EURASIP Journal on Applied Signal Processing, 6:886-901, 2004.
• 15: C.G.M. Snoek, M. Worring, J.M. Geusebroek, D.C. Koelma, and F.J. Seinstra. The mediamill trecvid 2004 semantic video search engine. In TREC Video Retrieval Evaluation Online Proceedings, 2004.
• 16: G.C. Stein, J. Rittscher, and A. Hoogs. Enabling video annotatation using a semantic database extended with visual knowledge. In Proceedings of ICME, 2003.
• 17: M. van Assem, M.R. Menken, A.Th. Schreiber, J. Wielemaker, and B. Wielinga. A method for converting thesauri to rdf/owl. In ISWC, 2004.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL

General Terms:
Documentation, Standardization

Keywords:
multimedia retrieval, ontologies