Experiments in social data mining: The TopicShop system

ABSTRACT

Social data mining systems enable people to share opinions and benefit from each other's experience. They do this by mining and redistributing information from computational records of social activity such as Usenet messages, system usage history, citations, or hyperlinks. Some general questions for evaluating such systems are: (1) is the extracted information valuable? and (2) do interfaces based on the information improve user task performance? We report here on TopicShop, a system that mines information from the structure and content of Web pages and provides an exploratory information workspace interface. We carried out experiments that yielded positive answers to both evaluation questions. First, a number of automatically computable features about Web sites do a good job of predicting expert quality judgments about sites. Second, compared to popular Web search interfaces, the TopicShop interface to this information lets users select significantly more high-quality sites, in less time and with less effort, and to organize the sites they select into personally meaningful collections more quickly and easily. We conclude by discussing how our results may be applied and considering how they touch on general issues concerning quality, expertise, and consensus.

REFERENCES

• 1: David Abrams , Ron Baecker , Mark Chignell, Information archiving with bookmarks: personal Web space construction and organization, Proceedings of the SIGCHI conference on Human factors in computing systems, p.41-48, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274651]
• 2: Charu C. Aggarwal , Joel L. Wolf , Kun-Lung Wu , Philip S. Yu, Horting hatches an egg: a new graph-theoretic approach to collaborative filtering, Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data mining, p.201-212, August 15-18, 1999, San Diego, California, United States [doi>10.1145/312129.312230]
• 3: Brian Amento , Loren Terveen , Will Hill, Does “authority” mean quality? predicting expert quality ratings of Web documents, Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.296-303, July 24-28, 2000, Athens, Greece [doi>10.1145/345508.345603]
• 4: Brian Amento , Loren Terveen , Will Hill , Deborah Hix, TopicShop: enhanced support for evaluating and organizing collections of Web sites, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.201-209, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354771]
• 5: Brian Amento , Will Hill , Loren Terveen , Deborah Hix , Peter Ju, An empirical evaluation of user interfaces for topic management of Web sites, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.552-559, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303156]
• 6: Michelle Q. Wang Baldonado , Terry Winograd, SenseMaker: an information-exploration interface supporting the contextual evolution of a user's interests, Proceedings of the SIGCHI conference on Human factors in computing systems, p.11-18, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258563]
• 7: Bederson, B. B., Hollan, J. D., Perlin, K., Meyer, J., Bacon, D., and Furnas, G. 1996. Pad++: A zoomable graphical sketchpad for exploring alternate interface physics. J. Visual Lang. Comput. 7, 3--31.
• 8: Krishna Bharat , Monika R. Henzinger, Improved algorithms for topic distillation in a hyperlinked environment, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.104-111, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.290972]
• 9: Daniel Billsus , Michael J. Pazzani, Learning Collaborative Information Filters, Proceedings of the Fifteenth International Conference on Machine Learning, p.46-54, July 24-27, 1998
• 10: Chris Buckley, Implementation of the SMART Information Retrieval System, Cornell University, Ithaca, NY, 1985
• 11: Bush, V. 1945. As we may think. The Atlantic Monthly (July).
• 12: Stuart K. Card , George G. Robertson , Jock D. Mackinlay, The information visualizer, an information workspace, Proceedings of the SIGCHI conference on Human factors in computing systems: Reaching through technology, p.181-186, April 27-May 02, 1991, New Orleans, Louisiana, United States [doi>10.1145/108844.108874]
• 13: Stuart K. Card , George G. Robertson , William York, The WebBook and the Web Forager: an information workspace for the World-Wide Web, Proceedings of the SIGCHI conference on Human factors in computing systems: common ground, p.111-ff., April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/238386.238446]
• 14: Soumen Chakrabarti , Byron Dom , Prabhakar Raghavan , Sridhar Rajagopalan , David Gibson , Jon Kleinberg, Automatic resource compilation by analyzing hyperlink structure and associated text, Computer Networks and ISDN Systems, v.30 n.1-7, p.65-74, April 1, 1998
• 15: Matthew Chalmers , Kerry Rodden , Dominique Brodbeck, The order of things: activity-centered information access, Proceedings of the seventh international conference on World Wide Web 7, p.359-367, April 1998, Brisbane, Australia
• 16: Andrew Crossen , Jay Budzik , Kristian J. Hammond, Flytrap: intelligent group music recommendation, Proceedings of the 7th international conference on Intelligent user interfaces, January 13-16, 2002, San Francisco, California, USA [doi>10.1145/502716.502748]
• 17: Egghe, L. and Rousseau, R. 1990. Introduction to Informetrics: Quantitative Methods in Library, Documentation, and Information Science. Elsevier, New York.
• 18: Garfield, E. 1979. Citation Indexing. ISI, Philadelphia.
• 19: David Goldberg , David Nichols , Brian M. Oki , Douglas Terry, Using collaborative filtering to weave an information tapestry, Communications of the ACM, v.35 n.12, p.61-70, Dec. 1992 [doi>10.1145/138859.138867]
• 20: Ron R. Hightower , Laura T. Ring , Jonathan I. Helfman , Benjamin B. Bederson , James D. Hollan, Graphical multiscale Web histories: a study of padprints, Proceedings of the ninth ACM conference on Hypertext and hypermedia : links, objects, time and space---structure in hypermedia systems: links, objects, time and space---structure in hypermedia systems, p.58-65, June 20-24, 1998, Pittsburgh, Pennsylvania, United States [doi>10.1145/276627.276634]
• 21: Hill, W. C. and Hollan, J. D. 1994. History-enriched digital objects: Prototypes and policy issues. Inf. Soc. 10, 2, 139--145.
• 22: Will Hill , Loren Terveen, Using frequency-of-mention in public conversations for social filtering, Proceedings of the 1996 ACM conference on Computer supported cooperative work, p.106-112, November 16-20, 1996, Boston, Massachusetts, United States [doi>10.1145/240080.240229]
• 23: William C. Hill , James D. Hollan , Dave Wroblewski , Tim McCandless, Edit wear and read wear, Proceedings of the SIGCHI conference on Human factors in computing systems, p.3-9, May 03-07, 1992, Monterey, California, United States [doi>10.1145/142750.142751]
• 24: Will Hill , Larry Stead , Mark Rosenstein , George Furnas, Recommending and evaluating choices in a virtual community of use, Proceedings of the SIGCHI conference on Human factors in computing systems, p.194-201, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223929]
• 25: Jon M. Kleinberg, Authoritative sources in a hyperlinked environment, Proceedings of the ninth annual ACM-SIAM symposium on Discrete algorithms, p.668-677, January 25-27, 1998, San Francisco, California, United States
• 26: Joseph A. Konstan , Bradley N. Miller , David Maltz , Jonathan L. Herlocker , Lee R. Gordon , John Riedl, GroupLens: applying collaborative filtering to Usenet news, Communications of the ACM, v.40 n.3, p.77-87, March 1997 [doi>10.1145/245108.245126]
• 27: Jock D. Mackinlay , Ramana Rao , Stuart K. Card, An organic user interface for searching citation links, Proceedings of the SIGCHI conference on Human factors in computing systems, p.67-73, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223913]
• 28: Richard Mander , Gitta Salomon , Yin Yin Wong, A “pile” metaphor for supporting casual organization of information, Proceedings of the SIGCHI conference on Human factors in computing systems, p.627-634, May 03-07, 1992, Monterey, California, United States [doi>10.1145/142750.143055]
• 29: Catherine C. Marshall , Frank M. Shipman, III , James H. Coombs, VIKI: spatial hypertext supporting emergent structure, Proceedings of the 1994 ACM European conference on Hypermedia technology, p.13-23, September 19-23, 1994, Edinburgh, Scotland [doi>10.1145/192757.192759]
• 30: Alan J. Munro , D. J. Sanger , D. R. Benyon, Personal and Social Navigation of Information Space, Springer-Verlag, London, 1999
• 31: Deborah Barreau , Bonnie A. Nardi, Finding and reminding: file organization from the desktop, ACM SIGCHI Bulletin, v.27 n.3, p.39-43, July 1995 [doi>10.1145/221296.221307]
• 32: Page L., Brin, S., Motwani, R., and Winograd, T. 2002. The PageRank citation ranking: bringing order to the Web. Stanford Digital Libraries Working Paper.
• 33: Peter Pirolli , James Pitkow , Ramana Rao, Silk from a sow's ear: extracting usable structures from the Web, Proceedings of the SIGCHI conference on Human factors in computing systems: common ground, p.118-125, April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/238386.238450]
• 34: Peter Pirolli , Patricia Schank , Marti Hearst , Christine Diehl, Scatter/gather browsing communicates the topic structure of a very large text collection, Proceedings of the SIGCHI conference on Human factors in computing systems: common ground, p.213-220, April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/238386.238489]
• 35: James Pitkow , Peter Pirolli, Life, death, and lawfulness on the electronic frontier, Proceedings of the SIGCHI conference on Human factors in computing systems, p.383-390, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258805]
• 36: Paul Resnick , Hal R. Varian, Recommender systems, Communications of the ACM, v.40 n.3, p.56-58, March 1997 [doi>10.1145/245108.245121]
• 37: Paul Resnick , Neophytos Iacovou , Mitesh Suchak , Peter Bergstrom , John Riedl, GroupLens: an open architecture for collaborative filtering of netnews, Proceedings of the 1994 ACM conference on Computer supported cooperative work, p.175-186, October 22-26, 1994, Chapel Hill, North Carolina, United States [doi>10.1145/192844.192905]
• 38: George Robertson , Mary Czerwinski , Kevin Larson , Daniel C. Robbins , David Thiel , Maarten van Dantzich, Data mountain: using spatial memory for document management, Proceedings of the 11th annual ACM symposium on User interface software and technology, p.153-162, November 01-04, 1998, San Francisco, California, United States [doi>10.1145/288392.288596]
• 39: Romney, A. K., Weller, S. C., and Batchelder, W. H. 1986. Culture as consensus: A theory of culture and informant accuracy. Amer. Anthropol. 88, 2 (June), 313--338.
• 40: Upendra Shardanand , Pattie Maes, Social information filtering: algorithms for automating “word of mouth”, Proceedings of the SIGCHI conference on Human factors in computing systems, p.210-217, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223931]
• 41: Frank M. Shipman, III , Catherine C. Marshall , Mark LeMere, Beyond location: hypertext workspaces and non-linear views, Proceedings of the tenth ACM Conference on Hypertext and hypermedia : returning to our diverse roots: returning to our diverse roots, p.121-130, February 21-25, 1999, Darmstadt, Germany [doi>10.1145/294469.294498]
• 42: Marc A. Smith , Andrew T. Fiore, Visualization components for persistent conversations, Proceedings of the SIGCHI conference on Human factors in computing systems, p.136-143, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365073]
• 43: Terveen, L. G. and Hill, W. C. 2001. Beyond recommender systems: Helping people help each other. In HCI In The New Millennium, J. Carroll, Eds. Addison-Wesley, Reading, Mass.
• 44: Loren Terveen , Jessica McMackin , Brian Amento , Will Hill, Specifying preferences based on user history, Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/503376.503433]
• 45: Fernanda B. Viégas , Judith S. Donath, Chat circles, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.9-16, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.302981]
• 46: Alan Wexelblat , Pattie Maes, Footprints: history-rich tools for information foraging, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.270-277, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303060]
• 47: S. Whittaker , Q. Jones , L. Terveen, Managing Long Term Communications: Conversation and Contact Management, Proceedings of the 35th Annual Hawaii International Conference on System Sciences (HICSS'02)-Volume 4, p.115.2, January 07-10, 2002
• 48: Whittaker, S., Jones, Q., Nardi, B., Creech, M., Terveen, L., Isaacs, E., and Hainsworth, J. 2002. ContactMap: Using personal social networks to organize communication in a social desktop. Submitted.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.3 Group and Organization Interfaces
          Subjects: Collaborative computing

General Terms:
Design, Experimentation, Human Factors

Keywords:
Cocitation analysis, collaborative filtering, computer-supported cooperative work, information visualization, social filtering, social network analysis