Improving visual search with image segmentation

ABSTRACT

People's ability to accurately locate target objects in images is severely affected by the prevalence of the sought objects. This negative effect greatly impacts critical real world tasks, such as baggage screening and cell slide pathology, in which target objects are rare. We present three novel image presentation techniques that are designed to improve visual search. Our techniques rely on the images being broken into image segments, which are then recombined or displayed in novel ways. The techniques and their underlying design reasoning are described in detail, and three experiments are presented that provide initial evidence that these techniques lead to better search performance in a simulated cell slide pathology task.

REFERENCES

• 1: Adami, H., Pontén, J., Sparén, P., Bergström, R., Gustafsson, L.,&Friberg, L. (1994). Survival trend after invasive cervical cancer diagnosis in Sweden before and after cytologic screening. 1960--1984. Cancer, 73(1). pp 140--147.
• 2: AForge.NET. http://code.google.com/p/aforge/
• 3: Beck, M.,&Trafton, J. G. (2007). Local spatial layout consistency affects strategies but not memory during Visual Search. Journal of Vision, 7(9):1060.
• 4: Coltheart, V. (Ed.) (1999). Fleeting memories: Cognition of brief visual stimuli. MIT Press.
• 5: S R Ellis , L Stark, Statistical dependency in visual scanning, Human Factors, v.28 n.4, p.421-438, Aug. 1986
• 6: Fleck, M. S.,&Mitroff, S. R. (2007). Rare targets are rarely missed in correctable search. Psychological Science, 18, 943--947.
• 7: Clifton Forlines , Chia Shen , Daniel Wigdor , Ravin Balakrishnan, Exploring the effects of group size and display configuration on visual search, Proceedings of the 2006 20th anniversary conference on Computer supported cooperative work, November 04-08, 2006, Banff, Alberta, Canada [doi>10.1145/1180875.1180878]
• 8: Gilchrist, I. and Harvey, M. (2006). Evidence for a systematic component within scan paths in visual search. Visual Cognition, 14(4):704--715.
• 9: Groopman, J. 2008. How Doctors Think - Chapter 8: The Eye of the Beholder. Houghton Mifflin Company, New York. ISBN-13:978-0-618-61003-7.
• 10: Tim Halverson , Anthony J. Hornof, A minimal model for predicting visual search in human-computer interaction, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240693]
• 11: Tim Halverson, Integrating models of human-computer visual interaction, CHI '06 extended abstracts on Human factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1125451.1125779]
• 12: Anthony J. Hornof , Tim Halverson, Cognitive strategies and eye movements for searching hierarchical computer displays, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642656]
• 13: Keenlyside, R. A., Collins, C. L., Hancock, J. S., Gagnon, M. C., Cohn, R. D., Menoff, A. L., Dodd, L. G., Kurtycz, D. F., Hearn, T. L.,&Baker, Jr. E. L. (1999). Do proficiency test results correlate with the work performance of screeners who screen Papanicolaou smears? American Journal of Clinical Pathology, 112(6). pp. 769--776.
• 14: Krendel, E. S.,&Wodinsky, J. (1960). Search in an unstructured visual Feld. Journal of the Optical Society of America, 50, 562--568.
• 15: Claudia Mello-Thoms , Calvin F Nodine , Harold L Kundel, What attracts the eye to the location of missed and reported breast cancers?, Proceedings of the 2002 symposium on Eye tracking research & applications, March 25-27, 2002, New Orleans, Louisiana [doi>10.1145/507072.507095]
• 16: Pomplun M. (2006) Saccadic selectivity in complex visual search displays. Vision Research. Jun;46(12):1886--1900.
• 17: Pham, D., Xu, C.,&Prince, J. (2000). Current methods in medical image segmentation. Annual Review of Biomedical Engineering. pp. 15--37.
• 18: Raymond, J., Shapiro, K., Arnell, K. (1992). Temporary suppression of visual processing in an RSVP task: an attentional blink?. Journal of Experimental psychology. Human perception and performance, 18 (3): 84960.
• 19: Renshaw, A. (2001). Measuring and Reporting Errors in Surgical Pathology. American Journal of Clinical Pathology, 115. pp 338--341.
• 20: Jos B. T. M. Roerdink , Arnold Meijster, The watershed transform: definitions, algorithms and parallelization strategies, Fundamenta Informaticae, v.41 n.1-2, p.187-228, Jan. 2000
• 21: Scinto, L., Pillalamarri, R.,&Karsh, R. (1986). Cognitive strategies for visual search. Acta Psychologica, 62, 263--292.
• 22: Simonin, J., Kieffer, S.,&Carbonell, N. (2005). Effects of display layout on gaze activity during visual search. In Proc. of INTERACT Conference. pp. 1054--1058.
• 23: Robert Spence, Rapid, serial and visual: a presentation technique with potential, Information Visualization, v.1 n.1, p.13-19, March 2002 [doi>10.1057/palgrave/ivs/9500008]
• 24: Kent Wittenburg , Clifton Forlines , Tom Lanning , Alan Esenther , Shigeo Harada , Taizo Miyachi, Rapid serial visual presentation techniques for consumer digital video devices, Proceedings of the 16th annual ACM symposium on User interface software and technology, p.115-124, November 02-05, 2003, Vancouver, Canada [doi>10.1145/964696.964709]
• 25: Wolfe, J. M. (1998). Visual search. In H. Pashler (Ed.), Attention, East Sussex, UK: Psychology Press.
• 26: Wolfe, J. M., Horowitz, T. S., Van Wert, M. J., Kenner, N. M., Place, S. S.,&Kibbi, N. (2007). Low target prevalence is a stubborn source of errors in visual search tasks. Journal of Experimental Psychology: General.
• 27: Wolfe, J. M., Horowitz, T. S.&Kenner, N. (2005) Rare items often missed in visual searches. Nature, 435. pp. 439--440.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Graphical user interfaces (GUI)

General Terms:
Human Factors

Keywords:
image presentation, rsvp, segmentation, visual search