Extracting contextual information from multiuser systems for improving annotation-based retrieval of image data

ABSTRACT

In this paper, we present an approach for incorporating contextual knowledge into a multiuser image retrieval system which is based on annotations. Although the most existing keyword-based systems are expanded by conceptual knowledge (e.g. ontologies) modeling the topics in which the user is interested in, there still remain some unresolved problems, like existing differences in interpretation of image contents or inconsistencies in keyword assignments among different users. In our approach, multiple sources of information which are modeled as different annotation ontologies are brought together in order to extract contextual information, and thus attenuate users' subjectivity in content description. Finally, we evaluate our introduced approach on a real data set of sports images. The experiments show that our approach provides considerable retrieval quality, already in the first search iteration, which makes an additional query refinement dispensable. The results can even be further improved by applying lexical analysis for strings and error elimination methods.

REFERENCES

• 1: Ricardo A. Baeza-Yates , Berthier Ribeiro-Neto, Modern Information Retrieval, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1999
• 2: T. Berners-Lee, J. Hendler, and O. Lassila. The Semantic Web. In Scientific American, page 279, 2001.
• 3: D. Brickley and R. V. Guha. Resource Description Framework (RDF)Schema Specification. World Wide Web Consortium. 2000.
• 4: P.-J. Cheng and L.-F. Chien. Effective Image Annotation for Search using Multi-level Semantics. In Proceedings of International Conference of Asian Digital Libraries, pages 230--242. Springer, 2003.
• 5: AnHai Doan , Jayant Madhavan , Pedro Domingos , Alon Halevy, Learning to map between ontologies on the semantic web, Proceedings of the 11th international conference on World Wide Web, May 07-11, 2002, Honolulu, Hawaii, USA [doi>10.1145/511446.511532]
• 6: P. Duygulu , Kobus Barnard , J. F. G. de Freitas , David A. Forsyth, Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary, Proceedings of the 7th European Conference on Computer Vision-Part IV, p.97-112, May 28-31, 2002
• 7: L. Hollink, G. Schreiber, J. Wielemaker, and B. Wielinga. Semantic Annotation of Image Collections. In Proceedings of the Workshop on Knowledge Markup and Semantic Annotation, 2003.
• 8: T. S. Huang, Y. Rui, M. Ortega, and S. Mehrotra. Relevance Feedback: A Power Tool for Interactive Content-Based Image Retrieval.IEEE Transactions on Circuits and Systems for Video Technology, pages 25--36, 1998.
• 9: Yoshiharu Ishikawa , Ravishankar Subramanya , Christos Faloutsos, MindReader: Querying Databases Through Multiple Examples, Proceedings of the 24rd International Conference on Very Large Data Bases, p.218-227, August 24-27, 1998
• 10: O. Lassila and R. Swick. Resource Description Framework (RDF) Model and Syntax Specification. Technical report, W3C, February 1999.
• 11: S. McDonald. A Context-based Model of Semantic Similarity. ACM Transactions on Programming Languages and Systems (TOPLAS), 15(5):795--825, November 1997.
• 12: C. J. Van Rijsbergen, Information Retrieval, Butterworth-Heinemann, Newton, MA, 1979
• 13: Y. Rui, T. S. Huang, and S. Mehrotra. Content-Based Image Retrieval with Relevance Feedback in MARS. In Proceedings of the 1997 International Conference on Image Processing (ICIP'97), pages 815--818, 1997.
• 14: A. Th. (Guus) Schreiber , Barbara Dubbeldam , Jan Wielemaker , Bob Wielinga, Ontology-Based Photo Annotation, IEEE Intelligent Systems, v.16 n.3, p.66-74, May 2001 [doi>10.1109/5254.940028]
• 15: J. Torres, A. Parkes, and L. Corte-Real. Region-Based Relevance Feedback in Concept-Based Image Retrieval. In Proceedings of the 5th International Workshop on Image Analysis for Multimedia Interactive Services, Lisboa, Portugal, 2004.
• 16: J. Vompras and S. Conrad. A Semi-automated Framework for Supporting Semantic Image Annotation. In Proceedings of 5th International Workshop on Knowledge Markup and Semantic Annotation (SemAnnot 2005), pages 105--110, 2005.
• 17: L. Wenyin, S. Dumais, Y. Sun, H. Zhang, M. Czerwinski, and B. Field.Semi-Automatic Image Annotation. In Proceedings International Conference on Human-Computer Interaction (INTERACT'01), pages 326--333, 2001.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

Additional Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.4 Systems
          Subjects: Multimedia databases

General Terms:
Algorithms, Human Factors

Keywords:
image retrieval, personalized retrieval, semantic image annotation