Divide and conquer: the role of trust and assurance in the design of secure socio-technical systems

ABSTRACT

In order to be effective, secure systems need to be both correct (i.e. effective when used as intended) and dependable (i.e. actually being used as intended). Given that most secure systems involve people, a strategy for achieving dependable security must address both people and technology. Current research in Human-Computer Interactions in Security (HCISec) aims to increase dependability of the human element by reducing mistakes (e.g. through better user interfaces to security tools). We argue that a successful strategy also needs to consider the impact of social interaction on security, and in this respect trust is a central concept. We compare the understanding of trust in secure systems with the more differentiated models of trust in social science research. The security definition of "trust" turns out to map onto strategies that would be correctly described as "assurance" in the more differentiated model. We argue that distinguishing between trust and assurance yields a wider range of strategies for ensuring dependability of the human element in a secure socio-technical system. Furthermore, correctly placed trust can also benefit an organisation's culture and performance. We conclude by presenting design principles to help security designers decide "when to trust" and "when to assure", and give examples of how both strategies would be implemented in practice.

REFERENCES

• 1: Anne Adams , Martina Angela Sasse, Users are not the enemy, Communications of the ACM, v.42 n.12, p.40-46, Dec. 1999 [doi>10.1145/322796.322806]
• 2: Ross Anderson, Why cryptosystems fail, Proceedings of the 1st ACM conference on Computer and communications security, p.215-227, November 03-05, 1993, Fairfax, Virginia, United States [doi>10.1145/168588.168615]
• 3: Ross J. Anderson, Security Engineering: A Guide to Building Dependable Distributed Systems, John Wiley & Sons, Inc., New York, NY, 2001
• 4: Michael Bacharach , Diego Gambetta, Trust as type detection, Trust and deception in virtual societies, Kluwer Academic Publishers, Norwell, MA, 2001
• 5: Dixie B. Baker, Fortresses built upon sand, Proceedings of the 1996 workshop on New security paradigms, p.148-153, September 17-20, 1996, Lake Arrowhead, California, United States [doi>10.1145/304851.304886]
• 6: Sacha Brostoff , M. Angela Sasse, Safe and sound: a safety-critical approach to security, Proceedings of the 2001 workshop on New security paradigms, September 10-13, 2001, Cloudcroft, New Mexico [doi>10.1145/508171.508178]
• 7: Peter Checkland , Jim Scholes, Soft systems methodology in action, John Wiley & Sons, Inc., New York, NY, 1990
• 8: Paul Dourish , David Redmiles, An approach to usable security based on event monitoring and visualization, Proceedings of the 2002 workshop on New security paradigms, September 23-26, 2002, Virginia Beach, Virginia [doi>10.1145/844102.844116]
• 9: Fukuyama, F. Social Capital and the Civil Society. 2nd Conference on Second Generation Reforms 1999. Washington, DC: IMF.
• 10: Simson Garfinkel , Gene Spafford, Practical Unix and Internet security (2nd ed.), O'Reilly & Associates, Inc., Sebastopol, CA, 1996
• 11: Handy, C. Trust and the Virtual Organization. Harvard Business Review 73(3) 1995. pp 40--50.
• 12: Ka-Ping, Y. User Interaction Design for Secure Systems. 2002. http://zesty.ca/sid
• 13: Kahn, D. The Codebreakers. 1967. Macmillan.
• 14: Thomas K. Landauer, Trouble with Computers: Usefulness, Usability, and Productivity, MIT Press, Cambridge, MA, 1996
• 15: Mayer, R. C., Davis, J. H., & Schoorman, F. D. An Integrative Model of Organizational Trust. Academy of Management Review 1995. 20(3), pp 709--734.
• 16: McAllister, D. J. Affect- and Cognitation-based Trust as Foundations for Interpersonal Cooperation in Organizations. Academy of Management Journal 1995. 38(1), pp 24--59.
• 17: Kevin D. Mitnick , William L. Simon, The Art of Deception: Controlling the Human Element of Security, John Wiley & Sons, Inc., New York, NY, 2003
• 18: Poulsen, K. Mitnick to lawmakers: People, phones and weakest links. 2000. http://www.politechbot.com/p-00969.html
• 19: Putnam, R. D. Bowling Alone: The Collapse and Revival of American Community. 2000. New York: Simon & Schuster.
• 20: Resnick, P. Beyond Bowling Together: SocioTechnical Capital. HCI in the New Millenium 2002. pp 242--272. Boston, MA: Addison-Wesley.
• 21: Jens Riegelsberger , M. Angela Sasse , John D. McCarthy, The mechanics of trust: a framework for research and design, International Journal of Human-Computer Studies, v.62 n.3, p.381-422, March 2005 [doi>10.1016/j.ijhcs.2005.01.001]
• 22: Rousseau, D. M., Sitkin, S. B., Burt, R. S., & Camerer, C. Not so different after all: A cross-discipline view of trust. Academy of Management Review 1998. 23(3), pp 393--404.
• 23: Saltzer, J. H. & Schroeder, M. D. The protection of information in computer systems. IEEE 1975.
• 24: Sasse, M. A. Computer Security: Anatomy of a Usability Disaster, and a Plan for Recovery. CHI 2003 2003.
• 25: M. A. Sasse , S. Brostoff , D. Weirich, Transforming the 'Weakest Link' — a Human/Computer Interaction Approach to Usable and Effective Security, BT Technology Journal, v.19 n.3, p.122-131, July 2001 [doi>10.1023/A:1011902718709]
• 26: Schneier, B. Secrets and Lies. 2000. John Wiley & Sons.
• 27: Bruce Schneier, Beyond Fear: Thinking Sensibly about Security in an Uncertain World, Springer-Verlag New York, Inc., Secaucus, NJ, 2003
• 28: D. K. Smetters , R. E. Grinter, Moving from the design of usable security technologies to the design of useful secure applications, Proceedings of the 2002 workshop on New security paradigms, September 23-26, 2002, Virginia Beach, Virginia [doi>10.1145/844102.844117]
• 29: Dirk Weirich , Martina Angela Sasse, Pretty good persuasion: a first step towards effective password security in the real world, Proceedings of the 2001 workshop on New security paradigms, September 10-13, 2001, Cloudcroft, New Mexico [doi>10.1145/508171.508195]
• 30: Whitten, A. & Tygar, J. D. Why Johnny Can't Encrypt: A Usability Evaluation of PGP 5.0. Proceedings of the 8th USENIX Security Symposium, August 1999, Washington 1999.