Formative design evaluation of superbook

ABSTRACT

SuperBook is a hypertext browsing system designed to improve the usability of conventional documents. Successive versions of SuperBook were evaluated in a series of behavioral studies. Students searched for information in a statistics text. presented either in conventional printed form or in SuperBook form. The best version of SuperBook enabled students to answer search questions more quickly and accurately than they could with the conventional text. Students wrote higher quality “open-book” essays using SuperBook than they did with the conventional text, and their subjective ratings of the documentation strongly favored SuperBook. This work is a case study of formative design-evaluation. Behavioral evaluation of the first version of SuperBook showed how design factors and user strategies affected search and established baseline performance measures with printed text. The second version of SuperBook was implemented with the goal of improving search accuracy and speed. User strategies that had proved effective in the first study were made very easy and attractive to use. System response time for common operations was greatly improved. Behavioral evaluation of the new SuperBook demonstrated its superiority to printed text and suggested additional improvements that were incorporated into “MiteyBook,” a SuperBook implementation for PC-size screens. Search with MiteyBook proved to be approximately 25 percent faster and 25 percent more accurate than that obtained with a conventional printed book.

REFERENCES

• 1: ALLISON, D.R. Electronic publishing technologies {Special Issue}. IEEE Computer 21, 1 (1988).
• 2: BECKER, R. A., AND CHAMBERS, J. M. S An Interactive Environment for Data Analysis and Graphics. Wadsworth, Belmont, Calif., 1984.
• 3: BUSH, V. As we may think. Atlantic Monthly 76, 1 (1945), 101-108.
• 4: Cyril Cleverdon, Optimizing convenient online access to bibliographic databases, Information Services and Use, v.4 n.1-2, p.37-47, April 1984
• 5: Jeff Conklin, Hypertext: an introduction and survey, Computer, v.20 n.9, p.17-41, Sept. 1987 [doi>10.1109/MC.1987.1663693]
• 6: DEERWESTER, S., DUMAIS, S. T., FURNAS, G. W., LANDAUER, T. K., AND HARSHMAN, R. A. Indexing by latent semantic analysis. In press.
• 7: DUMAIS, S.T. Textual information retrieval. In Handbook of Human-Computer Interaction, M. Helander, Ed. Elsevier Science Publishers B. V. (North-Holland), Amsterdam, 1988.
• 8: S. T. Dumais , G. W. Furnas , T. K. Landauer , S. Deerwester , R. Harshman, Using latent semantic analysis to improve access to textual information, Proceedings of the SIGCHI conference on Human factors in computing systems, p.281-285, May 15-19, 1988, Washington, D.C., United States [doi>10.1145/57167.57214]
• 9: EGAN, D.E. Electronic Text Browsers {Special Issue}. In press.
• 10: EGAN, D. E., REMDE, J. R., LANDAUER, T. K., LOCHBAUM, C. C., AND GOMEZ, L.M. Acquiring information in books and SuperBooks. In press.
• 11: ELKERTON, J., AND WILLIGES, R.C. Information retrieval strategies in a file-search environment. Hum. Factors 26 (1984), 171-184.
• 12: ENGELBART, D. C., AND ENGLISH, W.K. A research center for augmenting human intellect. In Proceedings of the 1968 Fall Joint Computer Con{erence (San Francisco, Calif., Dec. 9-11). AFIPS Press, Montvale, N.J., 1968, pp. 395-410.
• 13: George W. Furnas, Experience with an adaptive indexing scheme, Proceedings of the SIGCHI conference on Human factors in computing systems, p.131-135, April 1985, San Francisco, California, United States
• 14: G. W. Furnas, Generalized fisheye views, Proceedings of the SIGCHI conference on Human factors in computing systems, p.16-23, April 13-17, 1986, Boston, Massachusetts, United States
• 15: FURNAS, G. W., LANDAUER, T. K., GOMEZ, L. M., AND DUMAIS, S.T. Statistical semantics: Analysis of the potential performance of key-word information systems. Bell Sys. Tech. J. 62, 6 (1983), 1753-1806.
• 16: GOMEZ, L. M., AND LOCHBAUM, C.C. People can retrieve more objects with enriched key-word vocabularies. But is there a human performance cost? In Human-Computer Interaction--Interact '84, B. Shackel, Ed. North-Holland, Amsterdam, 1984, pp. 257-261.
• 17: GOMEZ, L. M., LOCHBAUM, C. C., AND LANDAUER, T.K. All the right words: Finding what you want as a function of richness of indexing vocabulary. In press.
• 18: Michael D. Good , John A. Whiteside , Dennis R. Wixon , Sandra J. Jones, Building a user-derived interface, Communications of the ACM, v.27 n.10, p.1032-1043, Oct 1984 [doi>10.1145/358274.358284]
• 19: GORDON, S., GUSTAVEL, J., MOORE, J., AND HANKEY, J. The effects of hypertext on reader knowledge representation. In Proceedings of the Human Factors Society--32nd Annual Meeting, Human Factors Society, Santa Monica, Calif., 1988, pp. 296-300.
• 20: GOULD, J. D. How to design usable systems. In Handbook of Human-Computer Interaction, M. Helander, Ed. Elsevier Science Publishers B. V. (North-Holland), Amsterdam, 1988.
• 21: John D. Gould , Lizette Alfaro , Rich Finn , Brian Haupt , Angela Minuto, Why reading was slower from CRT displays than from paper, Proceedings of the SIGCHI/GI conference on Human factors in computing systems and graphics interface, p.7-11, April 05-09, 1987, Toronto, Ontario, Canada
• 22: Wilfred J. Hansen , Christina Haas, Reading and writing with computers: a framework for explaining differences in performance, Communications of the ACM, v.31 n.9, p.1080-1089, Sept. 1988 [doi>10.1145/48529.48532]
• 23: MANTEI, M.M. A study of disorientation behavior in ZOG. Ph.D. dissertation, University of Southern California, Dept. of Computer Science, Los Angeles, 1982.
• 24: Gary Marchionini , Ben Shneiderman, Finding Facts vs. Browsing Knowledge in Hypertext Systems, Computer, v.21 n.1, p.70-80, January 1988 [doi>10.1109/2.222119]
• 25: Donald L. McCracken , Robert M. Akscyn, Experience with the ZOG human-computer interface system, International Journal of Man-Machine Studies, v.21 n.4, p.293-310, Oct. 1984 [doi>10.1016/S0020-7373(84)80050-4]
• 26: NELSON, T. H. Getting it out of our system. In Information Retrieval: A Critical Review, G. Schechter, Ed. Thompson Books, Washington, D.C., 1967.
• 27: PERLMAN, G. An overview of SAM: A hypertext interface to Smith & Mosier's. Tech. Rep. TR- 87-09. Wang Institute, Tyngsboro, Mass., 1987.
• 28: Joel R. Remde , Louis M. Gomez , Thomas K. Landauer, SuperBook: an automatic tool for information exploration—hypertext?, Proceeding of the ACM conference on Hypertext, p.175-188, November 1987, Chapel Hill, North Carolina, United States [doi>10.1145/317426.317440]
• 29: Ben Shneiderman, Designing the user interface: strategies for effective human-computer interaction, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1986
• 30: John,B. Smith , Stephen,F. Weiss, Hypertext, Communications of the ACM, v.31 n.7, p.816-819, July 1988 [doi>10.1145/48511.48512]
• 31: UHLER, S.A. Architecture and design of the MGR window system. Tech. Memo. TM-ARH- 013070. Bellcore, Morristown, N.J., 1988.
• 32: Andries van Dam, Hypertext '87: keynote address, Communications of the ACM, v.31 n.7, p.887-895, July 1988 [doi>10.1145/48511.48519]
• 33: VAN SICKEL, P., SIERZEGA, K. F., HERRING, C. A., AND FRUND, J.J. Documentation management for large systems of equipment. In Proceedings of RIAO '88 User-Oriented Content-Based Text and Image Handling. MIT, Cambridge, Mass., 1988, pp. 124-137.
• 34: Janet H. Walker, Document Examiner: delivery interface for hypertext documents, Proceeding of the ACM conference on Hypertext, p.307-323, November 1987, Chapel Hill, North Carolina, United States [doi>10.1145/317426.317448]
• 35: WATT, J.H. Level of abstraction structured text. IEEE Trans. Syst., Man Cybern. 18 (1988), 497-505.
• 36: WEYER, S. A. Searching for information in a dynamic book. Ph.D. dissertation, Stanford University, Palo Alto, Calif., 1982.
• 37: WEYER, S.A. The design of a dynamic book for information search. Int. J. Man Mach. Stud. 17 (1982), 87-107.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.4 Systems and Software

          Nouns: SuperBook

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.2 Design Tools and Techniques
          Subjects: User interfaces

  H. Information Systems
  H.1 MODELS AND PRINCIPLES
  H.3 INFORMATION STORAGE AND RETRIEVAL

General Terms:
Design, Documentation, Human Factors