The effects of source credibility ratings in a cultural heritage information aggregator

ABSTRACT

State of the art web search applications allow the user to aggregate information from many sources. Because of this, users are confronted with having to assess the reliability of information from different sources. This paper reports on an empirical user study on the effect of displaying credibility ratings of multiple cultural heritage sources (e.g. museum websites, art blogs) on users' search performance and selection. The study investigated whether source credibility has an in uence on users' search performance when they are confronted with only a few information sources or where there are many. The results of our online interactive study (n=122) show that by presenting the source credibility information explicitly, people's confidence in their selection of information significantly increases, even though it does not necessarily make search more time efficient. Additionally, we highlight credibility issues that are applicable beyond the cultural heritage domain, such as issues related to credibility measures and choice of visualization.

REFERENCES

• 1: Alia Amin , Jacco van Ossenbruggen , Lynda Hardman , Annelies van Nispen, Understanding cultural heritage experts' information seeking needs, Proceedings of the 8th ACM/IEEE-CS joint conference on Digital libraries, June 16-20, 2008, Pittsburgh PA, PA, USA [doi>10.1145/1378889.1378897]
• 2: D. K. Berlo, J. B. Lemert, and R. J. Mertz. Dimensions for Evaluating the Acceptability of Message Sources. Public Opin Q, 33(4):563--576, 1970.
• 3: W. P. Cassidy. Online news credibility: An examination of the perceptions of newspaper journalists. Journal of Computer-Mediated Communication, 12(2):478--498, 2007.
• 4: T. Chesney. An empirical examination of wikipedia's credibility. First Monday, 11(11), Nov. 2006.
• 5: J. H. Choi, J. H. Watt, and M. Lynch. Perceptions of news credibility about the war in iraq: Why war opponents perceived the internet as the most credible medium. Journal of Computer-Mediated Communication, 12(1):209--229, 2006.
• 6: M. Craigie, B. Loader, R. Burrows, and S. Muncer. Reliability of health information on the internet: An examination of experts' ratings. J Med Internet Res, 4(1):2, Jan 2002.
• 7: Henriette Cramer , Vanessa Evers , Satyan Ramlal , Maarten Someren , Lloyd Rutledge , Natalia Stash , Lora Aroyo , Bob Wielinga, The effects of transparency on trust in and acceptance of a content-based art recommender, User Modeling and User-Adapted Interaction, v.18 n.5, p.455-496, November 2008 [doi>10.1007/s11257-008-9051-3]
• 8: W. Du , B. Craig, and J. Cherry. Historians' use of archival sources: Promises and pitfalls of the digital age. Public Historian, 26(2):7--22, 2004.
• 9: M. S. Eastin. Credibility assessments of online health information: The effnects of source expertise and knowledge of content. Journal of Computer-Mediated Communication, 6(4):0{0, 2001.
• 10: Eero Hyvonen, et al. Culturesampo -- finnish cultural heritage collections on the semantic web 2.0. In Proc. of the 1st International Symposium on Digital humanities for Japanese Arts and Cultures (DH-JAC-2009), Ritsumeikan, Kyoto, Japan, March 2009.
• 11: G. Eysenbach and C. Kohler. How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews. BMJ, 324(7337):573--577, 2002.
• 12: Khairun Nisa Fachry , Jaap Kamps , Junte Zhang, Access to archival material in context, Proceedings of the second international symposium on Information interaction in context, October 14-17, 2008, London, United Kingdom [doi>10.1145/1414694.1414718]
• 13: A. J. Flanagin and M. J. Metzger. The role of site features, user attributes, and information verification behaviors on the perceived credibility of web-based information. New Media Society, 9(2):319--342, 2007.
• 14: B. J. Fogg , Hsiang Tseng, The elements of computer credibility, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.80-87, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303001]
• 15: W. E. Hick. On the rate of gain of information. The Quarterly Journal of Experimental Psychology, 4(1):11--26, 1952.
• 16: Traci Hong, The influence of structural and message features on Web site credibility, Journal of the American Society for Information Science and Technology, v.57 n.1, p.114-127, January 2006 [doi>10.1002/asi.v57:1]
• 17: Hovland, Carl I. and Weiss, Walter. The influence of source credibility on communication e ffectiveness. The Public Opinion Quarterly, 15(4):635--650, 1951.
• 18: S. Kiousis. Public trust or mistrust? perceptions of media credibility in the information age. Mass Communication & Society, 4(4):381--403, 2001.
• 19: Jürgen Koenemann , Nicholas J. Belkin, A case for interaction: a study of interactive information retrieval behavior and effectiveness, Proceedings of the SIGCHI conference on Human factors in computing systems: common ground, p.205-212, April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/238386.238487]
• 20: D. A. B. Lindberg and B. L. Humphreys. Medicine and Health on the Internet: The Good, the Bad, and the Ugly. JAMA, 280(15):1303--1304, 1998.
• 21: Rui Lopes , Luis Carriço, On the credibility of wikipedia: an accessibility perspective, Proceeding of the 2nd ACM workshop on Information credibility on the web, October 30-30, 2008, Napa Valley, California, USA [doi>10.1145/1458527.1458536]
• 22: Miriam J. Metzger , Andrew J. Flanagin , Lara Zwarun, College student web use, perceptions of information credibility, and verification behavior, Computers & Education, v.41 n.3, p.271-290, November 2003 [doi>10.1016/S0360-1315(03)00049-6]
• 23: Vanessa Murdock , Mounia Lalmas, Workshop on aggregated search, ACM SIGIR Forum, v.42 n.2, December 2008 [doi>10.1145/1480506.1480520]
• 24: G. Peterson, P. Aslani, and A. K. Williams. How do consumers search for and appraise information on medicines on the internet? a qualitative study using focus groups. J Med Internet Res, 5(4):e33, Dec 2003.
• 25: S. Y. Rieh and D. R. Danielson. Credibility: A multidisciplinary framework. In Annual Review of Information Science and Technology, pp. 307--364, 2007.
• 26: G. Schreiber, A. Amin, M. van Assem, V. de Boer, L. Hardman, M. Hildebrand, L. Hollink, Z. Huang, J. van Kersen, M. de Niet, B. Omelayenko, J. van Ossenbruggen, R. Siebes, J. Taekema, J. Wielemaker, and B. Wielinga. Multimedian e-culture demonstrator. In Proc. ISWC 2006, vol. 4273 of LNCS, pages 951--958. Springer, 2006.
• 27: S. S. Sundar and C. Nass. Conceptualizing sources in online news. Journal of Communication, 51(1):52--72, 2001.
• 28: Shawn Tseng , B. J. Fogg, Credibility and computing technology, Communications of the ACM, v.42 n.5, p.39-44, May 1999 [doi>10.1145/301353.301402]
• 29: Y. Tsfati and J. N. Cappella. Why do people watch news they do not trust? the need for cognition as a moderator in the association between news media skepticism and exposure. Media Psychology, 7(3):251--271, 2005.
• 30: B. J. Walther, Z. Wang, and T. Loh. The effect of top-level domains and advertisements on health web site credibility. J Med Internet Res, 6(3), Sep 2004.
• 31: Junte Zhang , Khairun Nisa Fachry , Jaap Kamps, Access to Archival Finding Aids: Context Matters, Proceedings of the 12th European conference on Research and Advanced Technology for Digital Libraries, September 14-19, 2008, Aarhus, Denmark [doi>10.1007/978-3-540-87599-4_63]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human factors

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.7 Digital Libraries
          Subjects: User issues

General Terms:
Design, Experimentation, Human Factors

Keywords:
credibility, cultural heritage, user study