Browsing and querying in online documentation: a study of user interfaces and the interaction process

ABSTRACT

A user interface study concerning the usage effectiveness of selected retrieval modes was conducted using an experimental text retrieval system, TeSS, giving access to online documentation of certain programming tools. Four modes of TeSS were compared: (1) browsing, (2) conventional boolean retrieval, (3) boolean retrieval based on Venn diagrams, and (4) these three combined. Further, the modes of TeSS were compared to the use of printed manuals. The subjects observed were 87 computing new to them. In the experiment the use of printed manuals is faster and provides answers of higher quality than any of the electronic modes. Therefore, claims about the effectiveness of computer-based text retrieval have to by vary in situations where printed manuals are manageable to the user. Among the modes of TeSS, browsing is the fastest and the one causing the fewest operational errors. On the same two variables, time and operational errors, the Venn diagram mode performs better than conventional boolean retrieval. The combined mode scores worst on the objective performance measures; nonetheless nearly all subject prefer this mode. Concerning the interaction process, the subjects tend to manage the complexities of the information retrieval tasks by issuing series of simple commands and exploiting the interactive capabilities of TeSS. To characterize the dynamics of the interaction process two concepts are introduced; threads and sequences of tactics. Threads in a query sequence describes the continuity during retrieval. Sequences of tactics concern the combined mode and describe how different retrieval modes succeed each other as the retrieval process evolves.

REFERENCES

• 1: Judith Avrahami , Yaakov Kareev, What do you expect to get when you ask for “a cup of coffee and a muffin or a croissant”?: on the interpretation of sentences containing multiple connectives, International Journal of Man-Machine Studies, v.38 n.3, p.429-434, March 1993 [doi>10.1006/imms.1993.1019]
• 2: BATES, M. J. 1986a. Subject access in online catalogs: A design model. J. Am. Soc. Inf. Sci. 37, 357-376.
• 3: BATES, M. J. 1986b. Terminological assistance for the online subject searcher. In Proceed-ings of the 2nd Conference on Computer Interfaces for Information Retrieval. Defense Technical Information Center, Cameron Station, Alexandria, Va., 7.
• 4: BATES, M. J. 1989. The design of browsing and berrypicking techniques for the online search interface. Online Rev. 13, 407-424.
• 5: Nicholas J. Belkin , W. Bruce Croft, Retrieval techniques, Annual review of information science and technology, vol. 22, Elsevier Science Inc., New York, NY, 1987
• 6: BELKIN,N.J.,ODDY,R.N.,AND BROOKS, H. M. 1982. ASK for information retrieval: Part I. Background and theory. J. Doc. 38, 61-71.
• 7: BLAIR, D. C. 1980. Searching biases in large interactive document retrieval systems. J. Am. Soc. Inf. Sci. 31, 271-277.
• 8: David C. Blair , M. E. Maron, An evaluation of retrieval effectiveness for a full-text document-retrieval system, Communications of the ACM, v.28 n.3, p.289-299, March 1985 [doi>10.1145/3166.3197]
• 9: C L Borgman, The user's mental model of an information retrieval system: an experiment on a prototype online catalog, International Journal of Man-Machine Studies, v.24 n.1, p.47-64, Jan. 1986 [doi>10.1016/S0020-7373(86)80039-6]
• 10: BORGMAN, C. L. 1986b. Why are online catalogs hard to use? Lessons learned from informa-tionretrieval studies. J. Am. Soc. Inf. Sci. 37, 387-400.
• 11: BROL~S,J.H.,FR~KJ~R, E., HERTZUM, M., LA ~ RUSDO ~ TTIR,M.K.,PILGAARD,K.B.,AND S~RENSEN, F. S. 1993. TeSS-projektet: Udvikling af et system til eksperimentel unders~gelse af brugergr~nseflader til edb-baseret teksts~gning. DIKU Rep. 93/18, Univ. of Copenhagen, Denmark.
• 12: Alan Dix , Janet Finley , Gregory Abowd , Russell Beale, Human-computer interaction (2nd ed.), Prentice-Hall, Inc., Upper Saddle River, NJ, 1998
• 13: Dennis E. Egan , Joel R. Remde , Louis M. Gomez , Thomas K. Landauer , Jennifer Eberhardt , Carol C. Lochbaum, Formative design evaluation of superbook, ACM Transactions on Information Systems (TOIS), v.7 n.1, p.30-57, Jan. 1989 [doi>10.1145/64789.64790]
• 14: T. R. Girill , Clement H. Luk, Hierarchical search support for hypertext on-line documentation, International Journal of Man-Machine Studies, v.36 n.4, p.571-585, April 1992 [doi>10.1016/0020-7373(92)90097-5]
• 15: HEARST, M. A. 1994. Using categories to provide context for full-text retrieval results. In Proceedings of RIAO'94. Intelligent Multimedia Information Retrieval Systems and Manage-ment. Rockefeller University, New York, 115-130.
• 16: Karl-Heinz Jerke , Peter Szabó , Arek Lesch , H. Röler , Thomas Schwab , Jürgen Herczeg, Combining hypermedia browsing with formal queries, Proceedings of the IFIP TC13 Third Interational Conference on Human-Computer Interaction, p.593-598, August 27-31, 1990
• 17: Jorgen Lindskov Knudsen , Boris Magnusson , Mats Lofgren , Ole L. Madsen, Object Oriented Software Development Environments: The Mjolner Approach, Prentice-Hall, Inc., Upper Saddle River, NJ, 1994
• 18: G. Marchionini, Making the transition from print to electronic encyclopaedias: adapation of mental models, International Journal of Man-Machine Studies, v.30 n.6, p.591-618, June 1989
• 19: Gary Marchionini , Ben Shneiderman, Finding Facts vs. Browsing Knowledge in Hypertext Systems, Computer, v.21 n.1, p.70-80, January 1988 [doi>10.1109/2.222119]
• 20: Charles T. Meadow , Barbara A. Cerny , Christine L. Borgman , Donald O. Case, Online access to knowledge: system design, Journal of the American Society for Information Science, v.40 n.2, p.86-98, March 1989 [doi>10.1002/(SICI)1097-4571(198903)40:2<86::AID-ASI3>3.0.CO;2-K]
• 21: MICHARD, A. 1982. Graphical presentation of boolean expressions in a database query language: Design notes and an ergonomic evaluation. Behav. Inf. Tech. 1, 279-288.
• 22: Carol Bergfeld Mills , Linda J. Weldon, Reading text from computer screens, ACM Computing Surveys (CSUR), v.19 n.4, p.329-357, Dec. 1987 [doi>10.1145/45075.46162]
• 23: Jakob Nielsen , Jonathan Levy, Measuring usability: preference vs. performance, Communications of the ACM, v.37 n.4, p.66-75, April 1994 [doi>10.1145/175276.175282]
• 24: Tadeusz Radecki, Trends in research on information retrieval—the potential for improvements in conventional Boolean retrieval systems, Information Processing and Management: an International Journal, v.24 n.3, p.219-227, May 1988 [doi>10.1016/0306-4573(88)90089-1]
• 25: SMITH, S. W. 1976. Venn diagramming for on-line searching. Spec. Libr. 67, 510-517.
• 26: SPOERRI, A. 1994. InfoCrystal: Integrating exact and partial matching approaches through visualization. In Proceedings of RIAO'94. Intelligent Multimedia Information Retrieval Systems and Management. Rockefeller University, New York, 687-696.
• 27: TVERSKY,A.AND KAHNEMAN, D. 1974. Judgment under uncertainty: Heuristics and biases. Science 185, 1124-1131.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Evaluation/methodology

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models; Query formulation
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Training, help, and documentation

General Terms:
Experimentation, Human Factors, Performance

Keywords:
documentation, interaction process, online manuals, usage effectiveness