Query definition using interactive saliency

ABSTRACT

Content-based image retrieval (CBIR) has been under investigation for a long time with many systems built to meet different application demands. However, in all systems, there is still a big gap between the user's expectation and the system's retrieval capabilities. Therefore, user interaction is an essential component of any CBIR system. Interaction up to now has mostly focused on global image features or similarities. We consider the interaction with salient details in the image i.e. points, lines, and regions. Interactive salient detail definition goes further than automatically summarizing the image into a set of salient details. We aim to dynamically update the user- and context-dependent definition of saliency based on relevance feedback from the user. In this paper, we propose an interaction framework for salient details from the perspective of the user.

REFERENCES

• 1: Alexander Dimai, Unsupervised Extraction of Salient Region-Descriptors for Content Based Image Retrieval, Proceedings of the 10th International Conference on Image Analysis and Processing, p.686, September 27-29, 1999
• 2: A. M. Martinez and J. R. Serra. A new approach to object-related image retrieval. Journal of Visual Languages and Computing, 11:345--363, 2000.
• 3: Arnold W. M. Smeulders , Marcel Worring , Simone Santini , Amarnath Gupta , Ramesh Jain, Content-Based Image Retrieval at the End of the Early Years, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.22 n.12, p.1349-1380, December 2000 [doi>10.1109/34.895972]
• 4: A. Th. (Guus) Schreiber , Barbara Dubbeldam , Jan Wielemaker , Bob Wielinga, Ontology-Based Photo Annotation, IEEE Intelligent Systems, v.16 n.3, p.66-74, May 2001 [doi>10.1109/5254.940028]
• 5: Cordelia Schmid , Roger Mohr, Local Grayvalue Invariants for Image Retrieval, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.19 n.5, p.530-535, May 1997 [doi>10.1109/34.589215]
• 6: Cordelia Schmid , Roger Mohr , Christian Bauckhage, Evaluation of Interest Point Detectors, International Journal of Computer Vision, v.37 n.2, p.151-172, June 2000 [doi>10.1023/A:1008199403446]
• 7: E. J. Pauwels , G. Frederix, Finding salient regions in images: nonparametric clustering for image segmentation and grouping, Computer Vision and Image Understanding, v.75 n.1-2, p.73-85, July/Aug. 1999 [doi>10.1006/cviu.1999.0763]
• 8: G. P. Nguyen and M. Worring. Interacting with salient details in content based image retrieval. ISIS technical report, 2003.
• 9: Jeroen Vendrig , Marcl Worring , Arnold W. M. Smeulders, Filter Image Browsing: Interactive Image Retrieval by Using Database Overviews, Multimedia Tools and Applications, v.15 n.1, p.83-103, September 2001 [doi>10.1023/A:1011367820253]
• 10: K. Mikolajczyk and C. Schmid. Indexing based on scale invariant interest points. IEEE International Conference on Computer Vision, pages 525--531, 2001.
• 11: K. N. Walker, T. F. Cootes, and C. J. Taylor. Locating salient object features. 9th British Machine Vision Conference, 2:557--566, 1998.
• 12: P. Kovesi. <u>http://www.cs.uwa.edu.au/pk/research/matlabfns/spatial/harris.m</u>.
• 13: Laurent Itti , Christof Koch , Ernst Niebur, A Model of Saliency-Based Visual Attention for Rapid Scene Analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.20 n.11, p.1254-1259, November 1998 [doi>10.1109/34.730558]
• 14: M. A. Hoang, J. Geusbroek, and A. W. M. Smeulders. Color texture measurement and segmentation. Proc. of the 2nd Intl. Workshop on Texture Analysis and Synthesis, 2002.
• 15: N. Sebe, Q. Tian, E. Loupias, M. S. Lew, and T. S. Huang. Salient points for content based image retrieval. IEEE Conference on Computer Vision and Pattern Recognition, 2001.
• 16: P. Bottoni, L. Cinque, S. Levialdi, and P. Mussio. Matching the resolution level to salient image features. Pattern Recognition, 31:98--104, 1998.
• 17: Q. Iqbal and J. K. Aggarwal. Retrieval by classification of images containing large manmade objects using perceptual grouping. Patter Recognition, 35:1463--1479, 2001.
• 18: Simone Santini , Amarnath Gupta , Ramesh Jain, Emergent Semantics through Interaction in Image Databases, IEEE Transactions on Knowledge and Data Engineering, v.13 n.3, p.337-351, May 2001 [doi>10.1109/69.929893]
• 19: Stéphane Bres , Jean-Michel Jolion, Detection of Interest Points for Image Indexation, Proceedings of the Third International Conference on Visual Information and Information Systems, p.427-434, June 02-04, 1999
• 20: Shyjan Mahamud , Karvel K. Thornber , Lance R. Williams, Segmentation of Salient Closed Contours from Real Images, Proceedings of the International Conference on Computer Vision-Volume 2, p.891, September 20-25, 1999
• 21: T. D. Alter , Ronen Basri, Extracting Salient Curves from Images: An Analysis of the Saliency Network, International Journal of Computer Vision, v.27 n.1, p.51-69, March, 1998 [doi>10.1023/A:1007953729443]
• 22: T. Gevers and A. Smeulders. The pictoseek www image search system. Proc. of the IEEE intl. conf. on Multimedia Computing and Systems, pages 264--269, 1999.
• 23: Tinne Tuytelaars , Luc J. Van Gool, Content-Based Image Retrieval Based on Local Affinely Invariant Regions, Proceedings of the Third International Conference on Visual Information and Information Systems, p.493-500, June 02-04, 1999
• 24: Yong-Sung Kim , Whoi-Yul Kim, Content-based trademark retrieval system using visually salient features, Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97), p.307, June 17-19, 1997

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

Keywords:
image retrieval, image segmentation, salient details