Take a look at the new version of this page: [ beta version ]. Tell us what you think.

ABSTRACT

Semantic video annotation using ontologies has received a large attention from the scientific community in the recent years. Ontologies are being regarded as an appropriate tool to bridge the semantic gap. In this paper we present an overview of the state-of-the-art of approaches and algorithms that exploit ontologies to perform semantic video annotation and present an approach to automatically learn rules describing high-level concepts. This approach exploits the domain knowledge embedded into an ontology to learn a set of rules for semantic video annotation. The proposed technique is an adaptation of the First Order Inductive Learner (FOIL) technique to the Semantic Web Rule Language (SWRL) standard: Experiments have been performed in two different video domains: i) the TRECVID 2005 broadcast news collection, to detect events related to airplanes, such as taxiing, flying, landing and taking off; ii) surveillance videos, to detect if a person enters or exits a specific area. The promising experimental performance demonstrates the effectiveness and flexibility of the proposed framework.

REFERENCES

• 1: Dublin Core Metadata Initiative - http://dublincore.org/.
• 2: TV Anytime Forum - http://www.tv-anytime.org/.
• 3: R. Arndt, R. Troncy, S. Staab, L. Hardman, and M. Vacura. Comm: Designing a well-founded multimedia ontology for the web. In Proc. of Int'l Semantic Web Conference, 2007.
• 4: A. D. Bagdanov, A. Del Bimbo, F. Dini, and W. Nunziati. Improving the robustness of particle filter-based visual trackers using online parameter adaptation. In Proc. of IEEE Int'l Conference on Advanced Video and Signal Based Surveillance, 2007.
• 5: Liang Bai , Songyang Lao , Gareth J. F. Jones , Alan F. Smeaton, Video Semantic Content Analysis based on Ontology, Proceedings of the International Machine Vision and Image Processing Conference, p.117-124, September 05-07, 2007 [doi>10.1109/IMVIP.2007.44]
• 6: Marco Bertini , Alberto Del Bimbo , Carlo Torniai , Costantino Grana , Rita Cucchiara, Dynamic pictorial ontologies for video digital libraries annotation, Workshop on multimedia information retrieval on The many faces of multimedia semantics, September 28-28, 2007, Augsburg, Bavaria, Germany [doi>10.1145/1290067.1290076]
• 7: S. Bloehdorn, K. Petridis, C. Saathoff, N. Simou, V. Tzouvaras, Y. Avrithis, S. Handschuh, I. Kompatsiaris, S. Staab, and M. Strintzis. Semantic annotation of images and videos for multimedia analysis. In Proc. of European Semantic Web Conference, 2005.
• 8: S. Castano, S. Espinosa, A. Ferrara, V. Karkaletsis, A. Kaya, S. Melzer, R. Moller, S. Montanelli, and G. Petasis. Ontology dynamics with multimedia information: The boemie evolution methodology. In Proc. of Int'l Workshop on Ontology Dynamics, 2007.
• 9: S. Dasiopoulou, V. Mezaris, I. Kompatsiaris, V. K. Papastathis, and M. G. Strintzis. Knowledge-assisted semantic video object detection. IEEE Trans. on Circuits and Systems for Video Technology, 15(10):1210--1224, 2005.
• 10: S. Dasiopoulou, C. Saathoff, P. Mylonas, Y. Avrithis, Y. Kompatsiaris, S. Staab, and M. Strintzis. Semantic Multimedia and Ontologies Theory and Applications, chapter Introducing Context and Reasoning in Visual Content Analysis: An Ontology-Based Framework, pages 99--122. Springer, 2008.
• 11: A. Dorado, J. Calic, and E. Izquierdo. A rule-based video annotation system. Circuits and Systems for Video Technology, IEEE Transactions on, 14(5):622--633, May 2004.
• 12: S. Ebadollahi, L. Xie, S.-F. Chang, and J. Smith. Visual event detection using multi-dimensional concept dynamics. In Proc. of IEEE Int'l Conference on Multimedia & Expo, 2006.
• 13: S. Espinosa Perald , A. Kaya , S. Melzer , R. Moller , M. Wessel, Towards a Media Interpretation Framework for the Semantic Web, Proceedings of the IEEE/WIC/ACM International Conference on Web Intelligence, p.374-380, November 02-05, 2007 [doi>10.1109/WI.2007.103]
• 14: Alexandre R. J. Francois , Ram Nevatia , Jerry Hobbs , Robert C. Bolles, VERL: An Ontology Framework for Representing and Annotating Video Events, IEEE MultiMedia, v.12 n.4, p.76-86, October 2005 [doi>10.1109/MMUL.2005.87]
• 15: R. Garcia and O. Celma. Semantic integration and retrieval of multimedia metadata. In Proc. of the Knowledge Markup and Semantic Annotation Workshop, 2005.
• 16: A. G. Hauptmann , M.-Y. Chen , M. Christel , W.-H. Lin , J. Yang, A Hybrid Approach to Improving Semantic Extraction of News Video, Proceedings of the International Conference on Semantic Computing, p.79-86, September 17-19, 2007 [doi>10.1109/ICSC.2007.3]
• 17: L. Hollink , S. Little , J. Hunter, Evaluating the application of semantic inferencing rules to image annotation, Proceedings of the 3rd international conference on Knowledge capture, October 02-05, 2005, Banff, Alberta, Canada [doi>10.1145/1088622.1088639]
• 18: L. Kennedy. Revision of LSCOM event/activity annotations, DTO challenge workshop on large scale concept ontology for multimedia. Advent technical report #221-2006-7, Columbia University, 2006.
• 19: M. Koskela, A. F. Smeaton, and J. Laaksonen. Measuring concept similarities in multimedia ontologies: Analysis and evaluation. IEEE Transactions on Multimedia, 9(5):912--922, August 2007.
• 20: Liza Leslie , Tat-Seng Chua , Jain Ramesh, Annotation of paintings with high-level semantic concepts using transductive inference and ontology-based concept disambiguation, Proceedings of the 15th international conference on Multimedia, September 25-29, 2007, Augsburg, Germany [doi>10.1145/1291233.1291335]
• 21: K.-H. Liu, M.-F. Weng, C.-Y. Tseng, Y.-Y. Chuang, and M.-S. Chen. Association and temporal rule mining for post-filtering of semantic concept detection in video. Multimedia, IEEE Transactions on, 10(2):240--251, Feb. 2008.
• 22: Nicolas Eric Maillot , Monique Thonnat, Ontology based complex object recognition, Image and Vision Computing, v.26 n.1, p.102-113, January, 2008 [doi>10.1016/j.imavis.2005.07.027]
• 23: Milind Naphade , John R. Smith , Jelena Tesic , Shih-Fu Chang , Winston Hsu , Lyndon Kennedy , Alexander Hauptmann , Jon Curtis, Large-Scale Concept Ontology for Multimedia, IEEE MultiMedia, v.13 n.3, p.86-91, July 2006 [doi>10.1109/MMUL.2006.63]
• 24: B. Neumann and R. Moeller. On scene interpretation with description logics. In Cognitive Vision Systems: Sampling the Spectrum of Approaches, LNCS, pages 247--278. Springer, 2006.
• 25: J. R. Quinlan, Learning Logical Definitions from Relations, Machine Learning, v.5 n.3, p.239-266, Aug. 1990 [doi>10.1023/A:1022699322624]
• 26: M.-L. Shyu, Z. Xie, M. Chen, and S.-C. Chen. Video semantic event/concept detection using a subspace-based multimedia data mining framework. Multimedia, IEEE Transactions on, 10(2):252--259, Feb. 2008.
• 27: C. Snoek, B. Huurnink, L. Hollink, M. de Rijke, G. Schreiber, and M. Worring. Adding semantics to detectors for video retrieval. 9(5):975--986, Aug. 2007.
• 28: C. Snoek and M. Worring. Multimedia event-based video indexing multimedia event-based video indexing using time intervals. IEEE Transactions on Multimedia, 7(4):638--647, 2005.
• 29: Chrisa Tsinaraki , Panagiotis Polydoros , Fotis Kazasis , Stavros Christodoulakis, Ontology-Based Semantic Indexing for MPEG-7 and TV-Anytime Audiovisual Content, Multimedia Tools and Applications, v.26 n.3, p.299-325, August 2005 [doi>10.1007/s11042-005-0894-x]
• 30: P. Viola and M. Jones. Rapid object detection using a boosted cascade of simple features. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, 2001.
• 31: Zheng-Jun Zha , Tao Mei , Zengfu Wang , Xian-Sheng Hua, Building a comprehensive ontology to refine video concept detection, Proceedings of the international workshop on Workshop on multimedia information retrieval, September 24-29, 2007, Augsburg, Bavaria, Germany [doi>10.1145/1290082.1290114]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Search process

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.10 Vision and Scene Understanding
            Subjects: Video analysis

General Terms:
Algorithms, Experimentation

Keywords:
events detection, learning rules, ontology, video retrieval