Search strategies in content-based image retrieval

ABSTRACT

This paper describes two studies that looked at users' ability to formulate visual queries with a Content-Based Image Retrieval system that uses dominant image colour as the primary indexing key. The first experiment examined users' performance with two visual search tools, a sketch tool and a structured browsing tool, with different types of image query. The results showed that while users were able to successfully search on the basis of colour, and were able to formulate visual queries, their ability to do so was affected by search task type. Search task type was also shown to be related to search tool choice. However, the results of study two showed that while users were able to complete all of the tasks, there was evidence to suggest that a degree of compromise was present in the users' choice of image that was largely due to problems relating to query formulation.

REFERENCES

• 1: Arnheim, R. (1974). Art and Visual Perception: A Psychology of the Creative Eye. The new version (The expanded and revised edition of the original publication of 1954). Berkeley: University of California Press.
• 2: Chad Carson , Serge Belongie , Hayit Greenspan , Jitendra Malik, Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.8, p.1026-1038, August 2002 [doi>10.1109/TPAMI.2002.1023800]
• 3: IEEE MultiMedia, The Holy Grail of Content-Based Media Analysis, IEEE MultiMedia, v.9 n.2, p.6-10, April 2002 [doi>10.1109/93.998041]
• 4: Conniss, L.R, Ashford, A J, Graham, M, E. (2000) Information Seeking Behaviour in Image Retrieval. Visor 1 Final Report. Library and Information Commission Research Report 95. Institute for Image Data Research, University of Northumbria.
• 5: Alberto Del Bimbo, Visual information retrieval, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1999
• 6: Eakins, J.P., Graham, M.E. Content-Based Image Retrieval. (1999) A Report to the JISC Technology Applications Programme. Institute for Image Data Research, University of Northumbria at Newcastle, UK. Available at http://www.unn.ac.uk/iidr/report.html
• 7: Enser, P.G.B (1995) Progress in Documentation: Pictorial information retrieval. Journal of Documentation, 51, 126--170.
• 8: Fidel R., (1997) Image Retrieval Task: Implications for the Design and Evaluation of Image Databases." The New Review of Hypermedia and Multimedia, 3:181--199.
• 9: Adrian Graham , Hector Garcia-Molina , Andreas Paepcke , Terry Winograd, Time as essence for photo browsing through personal digital libraries, Proceedings of the 2nd ACM/IEEE-CS joint conference on Digital libraries, July 14-18, 2002, Portland, Oregon, USA [doi>10.1145/544220.544301]
• 10: Amarnath Gupta , Simone Santini , Ramesh Jain, In search of information in visual media, Communications of the ACM, v.40 n.12, p.34-42, Dec. 1997 [doi>10.1145/265563.265570]
• 11: Joemon M. Jose , Jonathan Furner , David J. Harper, Spatial querying for image retrieval: a user-oriented evaluation, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.232-240, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.291000]
• 12: Lai, T.-S. CHROMA: A Photographic Image Retrieval System. Unpublished Ph.D. Thesis, University of Sunderland, UK, January 2000.
• 13: Gary Marchionini, Information seeking in electronic environments, Cambridge University Press, New York, NY, 1995
• 14: Markkula, M, and Sorumen, E. (1998) Searching for photos: Journalist's practices in pictorial IR. Proc. of CIR1998, First UK conference on Image Retrieval.
• 15: Sharon McDonald , Ting-Sheng Lai , John Tait, Evaluating a content based image retrieval system, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.232-240, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.383993]
• 16: Jakob Nielsen, Usability Engineering, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1995
• 17: Susanne Ornager, The newspaper image database: empirical supported analysis of users' typology and word association clusters, Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval, p.212-218, July 09-13, 1995, Seattle, Washington, United States [doi>10.1145/215206.215362]
• 18: Kerry Rodden , Wojciech Basalaj , David Sinclair , Kenneth Wood, Does organisation by similarity assist image browsing?, Proceedings of the SIGCHI conference on Human factors in computing systems, p.190-197, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365097]
• 19: Yossi Rubner , Carlo Tomasi, Perceptual Metrics for Image Database Navigation, Kluwer Academic Publishers, Norwell, MA, 2001
• 20: Exploratory image databases: content-based retrieval, Academic Press, Inc., Duluth, MN, 2001
• 21: Simone Santini , Ramesh Jain, Integrated Browsing and Querying for Image Databases, IEEE MultiMedia, v.7 n.3, p.26-39, July 2000 [doi>10.1109/93.879766]
• 22: Colin C. Venters , Richard J. Hartley , Matthew D. Cooper , William T. Hewitt, Query by Visual Example: Assessing the Usability of Content-Based Image Retrieval System User Interfaces, Proceedings of the Second IEEE Pacific Rim Conference on Multimedia: Advances in Multimedia Information Processing, p.514-521, October 24-26, 2001

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Search process

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Query formulation

General Terms:
Experimentation, Human Factors

Keywords:
content based image retrieval, visual search tools