Semantic maps and mobile context capturing for picture content visualization and management of picture databases

ABSTRACT

Modern cameras are powerful devices that can interoperate using international standards with other sensors that can capture the context of each picture taken. Future cameras are expected to integrate such sensors within them. The contextual parameters captured can be interpreted with the help of semantic maps which are maps that integrate semantic information on them. Semantic maps can be used for understanding and visualization of the picture contents as well as for the management of context and content in picture databases. In this paper, we present a model for the management of semantic image information and the contextual knowledge related to the image capture along with a system implementation that manages and exploits this knowledge. Digital picture contents in the database can be interrelated with, and understood based on, contextual semantic geospatial information that the system manages. We describe the integrated camera system, the calculation and visualization of the contextual parameters, and the management of the semantics of the spatial objects. Finally, we present the implementation of SPIM, a SPatial Image Management system that exploits semantic contextual knowledge, its retrieval functionality and the visual user interfaces.

REFERENCES

• 1: Christodoulakis S., Kontogiannis P., Petridis P., Moumoutzis N., Anastasiadis M. and Margazas T., A Distributed Multimedia Tourism Information System. In Information and Communication Technologies in Tourism 1997. Proceedings of the International Conference in Edinburgh, Scotland (ENTER 97). 295--306.
• 2: J. Ross Beveridge , Edward M. Riseman, How Easy is Matching 2D Line Models Using Local Search?, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.19 n.6, p.564-579, June 1997 [doi>10.1109/34.601245]
• 3: Boutell M., Luo J., Beyond Pixels: Exploiting Camera Metadata for Photo Classification, In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2004.
• 4: Exif 2002. Exif Version 2.2 Digital Still Camera Image File Format Standard, Japan Electronics and Information Technology Industries Assoc. http://www.exif.org/Exif2-2.pdf.
• 5: Google Earth 2007. Google Earth virtual globe. http://earth.google.com
• 6: Hollink L., Nguymen G., Schreiber G., Wielemaker J., Wielinga B., Worring M., Adding Spatial Semantics to Image Annotations, 4th International Workshop on Knowledge Markup and Semantic Annotation at ISWC'04.
• 7: Hollink L., Schreiber A. Th., Wielemaker J., Wielinga B. Semantic Annotation of Image Collections, In proceedings of the KCAP'03 Workshop on Knowledge Capture and Semantic Annotation, Florida, October 2003.
• 8: Franklin R., Ray C. K., Higher isn't Necessarily Better: Visibility Algorithms and Experiments, Advances in GIS Research: Proceedings of the 6th International Symposium on Spatial Data Handling, Waugh T. C., Healey R. G., (eds), Edinburgh, UK, pp751--770, 1994.
• 9: Gallanger A., Luo J., Hao W, Improved Blue Sky Detection Using Polynomial Model Fit, Proceedings of IEEE International Conference on Image Processing 2004, pp 2367--2370.
• 10: Jia Li , Shih-Fu Chang , Michael Lesk , Rainer Lienhart , Jiebo Luo , Arnold W. M. Smeulders, New challenges in multimedia research for the increasingly connected and fast growing digital society, Proceedings of the international workshop on Workshop on multimedia information retrieval, September 24-29, 2007, Augsburg, Bavaria, Germany [doi>10.1145/1290082.1290086]
• 11: NMEA 0183 2002. NMEA 0183 electrical and data specification for communication. http://www.nmea.org/pub/0183/index.html
• 12: Richard Nock , Frank Nielsen, Statistical Region Merging, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.26 n.11, p.1452-1458, November 2004 [doi>10.1109/TPAMI.2004.110]
• 13: John C. Platt, AutoAlbum: Clustering Digital Photographs using Probabilistic Model Merging, Proceedings of the IEEE Workshop on Content-based Access of Image and Video Libraries (CBAIVL'00), p.96, June 16-16, 2000
• 14: A. Th. (Guus) Schreiber , Barbara Dubbeldam , Jan Wielemaker , Bob Wielinga, Ontology-Based Photo Annotation, IEEE Intelligent Systems, v.16 n.3, p.66-74, May 2001 [doi>10.1109/5254.940028]
• 15: Tsinaraki C., Christodoulakis S., An MPEG-7 Query Language and a User Preference Model that allow Semantic Retrieval and Filtering of Multimedia Content, In ACM Verlag Multimedia Systems Journal, Special Issue on Semantic Multimedia Adaptation and Personalization, 13, 2, 2007, pp131--153.
• 16: Tsinaraki C, Fatourou E, Christodoulakis S., An Ontology-Driven Framework for the Management of Semantic Metadata describing Audiovisual Information, In Proc. of the Conference of Advanced Information Systems Engineering (CaiSE), Velden, Austria, 2003, pp340--356.
• 17: Chrisa Tsinaraki , Panagiotis Polydoros , Stavros Christodoulakis, Interoperability Support between MPEG-7/21 and OWL in DS-MIRF, IEEE Transactions on Knowledge and Data Engineering, v.19 n.2, p.219-232, February 2007 [doi>10.1109/TKDE.2007.33]
• 18: Vailaya, M., Fugueiredo, A., Jain A., Zhang, H., Image Classification for Content-based Indexing. IEEE Transactions on Image processing, 10, 1, Jan. 2001, pp117--129.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.7 Digital Libraries

General Terms:
Algorithms, Experimentation

Keywords:
GPS, GeoSpatial, camera, compass, context, image, multimedia, ontology, semantic, sensors