Toward automatic generation of image-text document surrogates to optimize cognition

ABSTRACT

The representation of information collections needs to be optimized for human cognition. Growing information collections play a crucial role in human experiences. While documents often include rich visual components, collections, including personal collections and those generated by search engines, are typically represented lists of text-only surrogates. By concurrently invoking complementary components of human cognition, combined image-text surrogates help people to more effectively see, understand, think about, and remember information collection. This research develops algorithmic methods that use the structural context of images in HTML documents to associate meaningful text and thus derive combined image-text surrogates.

REFERENCES

• 1: Glenberg, A.M., Langston, W.E., Comprehension of illustrated text: Pictures help to build mental models, J. Mem. Lang. 31(2) 1992.
• 2: Andruid Kerne , Eunyee Koh , Steven M. Smith , Andrew Webb , Blake Dworaczyk, combinFormation: Mixed-initiative composition of image and text surrogates promotes information discovery, ACM Transactions on Information Systems (TOIS), v.27 n.1, p.1-45, December 2008 [doi>10.1145/1416950.1416955]
• 3: National Science Foundation, Discoveries, http://www.nsf.gov/discoveries/, last visited 01/28/2009.
• 4: New York Public Library, Digital Collections, http://www.nypl.org/digital/collections_images.html, last visited 01/28/2009.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.0 GENERAL

General Terms:
Human Factors

Keywords:
information extraction, search representation, surrogates