Enhancing information scent: identifying and recommending quality tags

ABSTRACT

We describe a scenario of tag use and an empirical study of tags as socio-cognitive artifacts providing information scent. We articulated a three-step use scenario of tags, and used it to conceptualize tag "quality" as determined by use. We designed and conducted a user study to explore what attributes of tags and taggers predict the user-rated "quality" of tags. We found that frequency best predicted tag quality, while information entropy provided further refinement. We found that people rated our identified quality tags as higher in quality than general tags. But these identified quality tags were not perceived as better than self-generated tags. We derived a regression model for tag quality and discussed implications for social computing.

REFERENCES

• 1: Ed H. Chi , Todd Mytkowicz, Understanding the efficiency of social tagging systems using information theory, Proceedings of the nineteenth ACM conference on Hypertext and hypermedia, June 19-21, 2008, Pittsburgh, PA, USA [doi>10.1145/1379092.1379110]
• 2: Umer Farooq , Thomas G. Kannampallil , Yang Song , Craig H. Ganoe , John M. Carroll , Lee Giles, Evaluating tagging behavior in social bookmarking systems: metrics and design heuristics, Proceedings of the 2007 international ACM conference on Supporting group work, November 04-07, 2007, Sanibel Island, Florida, USA [doi>10.1145/1316624.1316677]
• 3: Michael J. Muller, Comparing tagging vocabularies among four enterprise tag-based services, Proceedings of the 2007 international ACM conference on Supporting group work, November 04-07, 2007, Sanibel Island, Florida, USA [doi>10.1145/1316624.1316676]
• 4: Shilad Sen , F. Maxwell Harper , Adam LaPitz , John Riedl, The quest for quality tags, Proceedings of the 2007 international ACM conference on Supporting group work, November 04-07, 2007, Sanibel Island, Florida, USA [doi>10.1145/1316624.1316678]
• 5: Shilad Sen , Shyong K. Lam , Al Mamunur Rashid , Dan Cosley , Dan Frankowski , Jeremy Osterhouse , F. Maxwell Harper , John Riedl, tagging, communities, vocabulary, evolution, Proceedings of the 2006 20th anniversary conference on Computer supported cooperative work, November 04-08, 2006, Banff, Alberta, Canada [doi>10.1145/1180875.1180904]
• 6: Peter Pirolli , Stuart Card, Information foraging in information access environments, Proceedings of the SIGCHI conference on Human factors in computing systems, p.51-58, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223911]
• 7: Pirolli, P. and Card, S. Information Foraging. Psychological Review, 106, (1999), 643--675.
• 8: Daniel M. Russell , Mark J. Stefik , Peter Pirolli , Stuart K. Card, The cost structure of sensemaking, Proceedings of the INTERACT '93 and CHI '93 conference on Human factors in computing systems, p.269-276, April 24-29, 1993, Amsterdam, The Netherlands [doi>10.1145/169059.169209]
• 9: John M. Carroll, Making use: a design representation, Communications of the ACM, v.37 n.12, p.28-35, Dec. 1994 [doi>10.1145/198366.198374]
• 10: Scott A. Golder , Bernardo A. Huberman, Usage patterns of collaborative tagging systems, Journal of Information Science, v.32 n.2, p.198-208, April 2006 [doi>10.1177/0165551506062337]
• 11: Cattuto, C., Baldassarri, A., Servedio, V.D.P. and Loreto, V. Vocabulary growth in collaborative tagging systems. Arxiv preprint arXiv:0704.3316, (2007).
• 12: Cattuto, C., Loreto, V. and Pietronero, L. Collaborative Tagging and Semiotic Dynamics. Arxiv preprint cs.CY/0605015, (2006).
• 13: Macgregor, G. and McCulloch, E. Collaborative tagging as a knowledge organisation and resource discovery tool. Library Review, 55, 5, (2006), 291--300.
• 14: Chi, E.H. and Mytkowicz, T. Understanding navigability of social tagging systems. In SigCHI alt.chi (2007).
• 15: Wai-Tat Fu, The microstructures of social tagging: a rational model, Proceedings of the ACM 2008 conference on Computer supported cooperative work, November 08-12, 2008, San Diego, CA, USA [doi>10.1145/1460563.1460600]
• 16: G. W. Furnas , T. K. Landauer , L. M. Gomez , S. T. Dumais, The vocabulary problem in human-system communication, Communications of the ACM, v.30 n.11, p.964-971, Nov. 1987 [doi>10.1145/32206.32212]
• 17: George W. Furnas , Caterina Fake , Luis von Ahn , Joshua Schachter , Scott Golder , Kevin Fox , Marc Davis , Cameron Marlow , Mor Naaman, Why do tagging systems work?, CHI '06 extended abstracts on Human factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1125451.1125462]
• 18: Shirky, C. Ontology is overrated. (2005). http://www.shirky.com/writings/ontologyoverrated.html.
• 19: Bush, V. As We May Think. Atlantic Monthly, 176, 1 (1945), 101--108.
• 20: Collins, A. M. and Loftus, E. F. A spreading-activation theory of semantic processing. Psychological Review, 82, 6, (1975), 407--428.
• 21: Donald A. Norman, Cognitive artifacts, Designing interaction: psychology at the human-computer interface, Cambridge University Press, New York, NY, 1991
• 22: Simon, H. A. A behavioral model of rational choice. Quarterly Journal of Economics, 69, (1955), 99--118.
• 23: Klein, G. and Klinger, D. Naturalistic Decision Making. Human Systems IAC Gateway, 11, 3, (1991), 16--19.
• 24: Weick, K. E. Sense-making in organizations. Sage Publications, Newbury Park, CA, 1996.
• 25: Wittgenstein, L. Philosophical Investigations. Blackwell Publishing, MA, 1953.
• 26: Berger, P. L. and Luckmann, T. The Social Construction of Reality: A treatise in the Sociology of Knowledge. Anchor Books, NY, 1966.
• 27: Harry Halpin , Valentin Robu , Hana Shepherd, The complex dynamics of collaborative tagging, Proceedings of the 16th international conference on World Wide Web, May 08-12, 2007, Banff, Alberta, Canada [doi>10.1145/1242572.1242602]
• 28: Paul Resnick , Hal R. Varian, Recommender systems, Communications of the ACM, v.40 n.3, p.56-58, March 1997 [doi>10.1145/245108.245121]
• 29: Vu, K. P. L., Hanley, G. L., Strybel, T. Z. and Proctor, R. W. Metacognitive Processes in Human-Computer Interaction: Self-Assessments of Knowledge as Predictors of Computer Expertise. International Journal of Human-Computer Interaction, 12, 1, (2000), 43--71.
• 30: David W. McDonald , Mark S. Ackerman, Expertise recommender: a flexible recommendation system and architecture, Proceedings of the 2000 ACM conference on Computer supported cooperative work, p.231-240, December 2000, Philadelphia, Pennsylvania, United States [doi>10.1145/358916.358994]
• 31: Paul Dourish , Victoria Bellotti, Awareness and coordination in shared workspaces, Proceedings of the 1992 ACM conference on Computer-supported cooperative work, p.107-114, November 01-04, 1992, Toronto, Ontario, Canada [doi>10.1145/143457.143468]
• 32: Carl Gutwin , Mark Roseman , Saul Greenberg, A usability study of awareness widgets in a shared workspace groupware system, Proceedings of the 1996 ACM conference on Computer supported cooperative work, p.258-267, November 16-20, 1996, Boston, Massachusetts, United States [doi>10.1145/240080.240298]
• 33: Jonathan Grudin, Why CSCW applications fail: problems in the design and evaluationof organizational interfaces, Proceedings of the 1988 ACM conference on Computer-supported cooperative work, p.85-93, September 26-28, 1988, Portland, Oregon, United States [doi>10.1145/62266.62273]
• 34: Kerry Rodden , Kenneth R. Wood, How do people manage their digital photographs?, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642682]
• 35: Slamecka, N. J. and Graf, P. The generation effect: Delineation of a phenomenon. Journal of Experimental Psychology, 4, 6, (1978), 592--604.
• 36: Melanie Tory , Torsten Moller, Evaluating Visualizations: Do Expert Reviews Work?, IEEE Computer Graphics and Applications, v.25 n.5, p.8-11, September 2005 [doi>10.1109/MCG.2005.102]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.3 Group and Organization Interfaces
          Subjects: Collaborative computing

General Terms:
Experimentation

Keywords:
quality tags, sense-making, social bookmarking