Accessible image search

ABSTRACT

There are about 8% of men and 0.8% of women suffering from colorblindness. We show that the existing image search techniques cannot provide satisfactory results for these users, since many images will not be well perceived by them due to the loss of color information. In this paper, we introduce a scheme named Accessible Image Search (AIS) to accommodate these users. Different from the general image search scheme that aims at returning more relevant results, AIS further takes into account the colorblind accessibilities of the returned results, i.e., the image qualities in the eyes of colorblind users. The scheme includes two components: accessibility assessment and accessibility improvement. For accessibility assessment, we introduce an analysisbased method and a learning-based method. Based on the measured accessibility scores, different reranking methods can be performed to prioritize the images with high accessibilities. In accessibility improvement component, we propose an efficient recoloring algorithm to modify the colors of the images such that they can be better perceived by colorblind users. We also propose the Accessibility Average Precision (AAP) for AIS as a complementary performance evaluation measure to the conventional relevance-based evaluation methods. Experimental results with more than 60,000 images and 20 anonymous colorblind users demonstrate the effectiveness and usefulness of the proposed scheme.

REFERENCES

• 1: Web Accessibility Initiative. http://www.w3.org/WAI/.
• 2: Vischeck. http://www.vischeck.com.
• 3: Accessibility research: aDesigner. http://www.research.ibm.com/trl/projects/acc tech/adesigner.htm
• 4: Google accessible search. http://labs.google.com/accessible/
• 5: H. Brettel, F. Vienot, and J. Mollon, "Computerized simulation of color appearance for dichromats", Journal of the Optical Society of America, vol. 14, no. 10, 1997
• 6: R. A. Fisher, "Statistical methods for research workers", Macmillan Pub Co, 1970
• 7: J.-B. Huang, Y.-C. Tseng, S.-I Wu, S.-J. Wang, "Information preserving color transformation for protanopia and deuteranopia", IEEE signal processing letter, vol. 14, no. 10, 2007
• 8: L. Jefferson and R. Harvey, "An interface to support color blind computer users", in Proceedings of ACM SIGCHI, 2007
• 9: L. Jefferson and R. Harvey, "Accommodating color blind computer users", in Proceedings of ACM Conference on Computers and Accessibility, 2006
• 10: G. Iaccarino, D. Malandrino, M. D. Percio, and V. Scarano, "Efficient edge-services for colorblind users", in Proceedings of International World Wide Web Conference, 2006
• 11: V. A. Kovalev, "Towards image retrieval for eight percent of color-blind men", in Proceedings of International Conference on Pattern Recognition, 2004
• 12: K. Rasche, R. Geist, and J. Westall, "Re-coloring images for gamuts of lower dimension", in Proceedings of Eurographics, 2005
• 13: A. J. Smola and Bernhard Scholkopf, "A tutorial on support vector machine", Statistics and computing, vol. 14, no. 3, 2004
• 14: K. Wakita and K. Shimamura, "Smart color: disambiguation framework for the colorblind", in Proceedings of ACM Conference on Computers and Accessibility, 2005
• 15: S. Yang and Y. M. Ro, "Visual content adaptation for color vision deficiency", in Proceedings of International Conference on Image Processing, 2003
• 16: CIE Technical Report, "Industrial colour-difference evaluation", 1995
• 17: Wandell, Foundations of Vision, Sunderland, MA:Sinauer, 1995
• 18: Wikipedia. www.wikipedia.org/wiki/Accessibility
• 19: K. Jarvelin and J. Kekalainen, "Cumulated gain-based evaluation of IR techniques", ACM transactions on Information Systems, vol. 20, no. 4, 2002
• 20: Z. Wang and A. C. Bovik, "Modern image quality assessment", Morgan & Claypool Publishers, 2006
• 21: Z. Lu, W. Lin, X. Yang, E. P. Ong, and S. Yao, "Modeling visual attention's modulatory aftereffects on visual sensitivity and quality evaluation", IEEE transactions on Image Processing, vol. 14, no. 11, 2005
• 22: M. S. Lew, N. Sebe, C. Djeraba, and R. Jain, "Content-based multimedia information retrieval: state of the art and challenges", ACM transactions on Multimedia Computing, Communications and Applications, 2006
• 23: M. Song, D. Tao, C. Chen, X. Li, and C.-W. Chen, "Colour to Grey: Visual Cue Preservation", IEEE Transactions on Pattern Analysis and Machine Intelligence, to appear
• 24: R. M. Evans, An introduction to color, Wiley, New York, 1948
• 25: IEEE CVPR 2005 Workshop on Computer Vision Applications for the Visually Impaired, http://www.soe.ucsc.edu/ manduchi/CVAVI/
• 26: ECCV 2008 Workshop on Computer Vision Applications for the Visually Impaired, http://www.ski.org/Rehab/Coughlan lab/General/CVAVI08.html
• 27: Y. Jing and S. Baluja, "PageRank for product image search", in Proceedings of International World Wide Web Conference, 2008
• 28: L. Kennedy and M. Naaman, "Generating diverse and representative image search results for landmarks", in Proceedings of International World Wide Web Conference, 2008
• 29: M. Hua, J. Pei, A. Fu, X. Lin, H.--F. Leung, "Efficiently answering top-k typicality queries on large databases," in Proceedings of VLDB, 2007
• 30: J.-B. Huang, S.-Y. Wu, and C.-S. Chen, "Enhancing color representation for the color vision impaired", in Proceedings of ECCV Workshop on Computer Vision Applications for the Visually Impaired, 2008
• 31: V. Vapnik. The nature of statistical learning theory, Springer, New York, 1995

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

Additional Classification:
  K. Computing Milieux
  K.4 COMPUTERS AND SOCIETY
      K.4.2 Social Issues
          Subjects: Assistive technologies for persons with disabilities

General Terms:
Algorithms, Experimentation

Keywords:
colorblindness, image search