Take a look at the new version of this page: [ beta version ]. Tell us what you think.

ABSTRACT

In previous work we laid out an approach to quantifying configuration complexity [3]. In that earlier work, we explicitly focused on complexity as experienced by expert systems managers, and thus looked at straight-line configuration procedures, ignoring the complexity faced by non-experts as they have to decide what configuration steps to follow. Decision complexity is the complexity faced by a non-expert system administrator---the person providing IT support in a small-business environment, who is confronted by decisions during the configuration process, and is a measure of how easy or hard it is to identify the appropriate sequence of configuration actions to perform in order to achieve a specified configuration goal. To identify spots of high decisionmaking complexity, we need a model of decision complexity for configuring and operating computing systems. This paper extends previous work on models and metrics for IT configuration complexity by adding the concept of decision complexity. As the first step towards a complete model of decision complexity, we describe an extensive user study of decision making in a carefully-mapped analogous domain (route planning), and illustrate how the results of that study suggest an initial model of decision complexity applicable to IT configuration. The model identifies the key factors affecting decision complexity and highlights several interesting results, including the fact that decision complexity has significantly different impacts on user-perceived difficulty than on objective measures like time and error rate. We also describe some of the implications of our decision complexity model for system designers seeking to automate the decision-making and reduce the configuration complexity of their systems.

REFERENCES

• 1: Aaron B. Brown , Joseph L. Hellerstein, An approach to benchmarking configuration complexity, Proceedings of the 11th workshop on ACM SIGOPS European workshop, September 19-22, 2004, Leuven, Belgium [doi>10.1145/1133572.1133609]
• 2: Aaron B. Brown , Joseph L. Hellerstein, Reducing the cost of IT operations: is automation always the answer?, Proceedings of the 10th conference on Hot Topics in Operating Systems, p.12-12, June 12-15, 2005, Santa Fe, NM
• 3: Brown, A. B., Keller, A., and Hellerstein, J. L. A model of configuration complexity and its application to a change management system. In Proceedings of the Ninth IFIP/IEEE International Symposium on Integrated Network Management (IM'05) (May 2005).
• 4: Crozier, R. W., and Svenson, O. Decision Making: Cognitive Models and Explanations, 1st ed. Routledge, Nov 1997.
• 5: David E Bell, H. R., and Tversky, A., Eds. Decision Making: Descriptive, Normative and Prescriptive Interactions. Cambridge University Press, Oct 19 1988.
• 6: Gadomski, A. M. Cognitive decision-making: Toga socio-cognitive approach. White paper, Aug 2006.
• 7: Galic, M., Halliday, A., Hatzikyriacos, A., Munaro, M., Parepalli, S., and Yang, D. A Secure Portal Using WebSphere Portal V5 and Tivoli Access Manager V4.1. Redbooks. IBM, Dec 2003.
• 8: Humphreys, J., and Turner, V. On-demand enterprises and utility computing: A current market assessment and outlook. Tech. Rep. 31513, IDC, July 2004.
• 9: Jerome, B. R., and Julie, S. C. A contribution of cognitive decision models to clinical assessment: Decomposing performance on the bechara gambling task. Psychological assessment 14 (2002).
• 10: Kahneman, D., and Tversky, A. Prospect theory: An analysis of decision under risk. Econometrica 47, 2 (1979), 263--91.
• 11: Of Government Commerce (UK), O. Best Practice for Service Support. IT Infrastructure Library Series. Stationery Office, June 2000.
• 12: Plous, S. The Psychology of Judgment and Decision Making, 1st ed. McGraw-Hill, Jan 1 1993.
• 13: Simon, H. A. Administrative Behavior, 4th ed. Free Press, March 1997.
• 14: Sternberg, R. J., and Ben-Zeev, T. Complex Cognition: The Psychology of Human Thought. Oxford University Press, Feb 2001.
• 15: Wickens, C. D., and Hollands, J. G. Engineering Psychology and Human Performance, 3rd ed. Prentice Hall, Sept 21 1999.

INDEX TERMS

Primary Classification:
  K. Computing Milieux
  K.6 MANAGEMENT OF COMPUTING AND INFORMATION SYSTEMS
      K.6.1 Project and People Management

Additional Classification:
  H. Information Systems
  H.4 INFORMATION SYSTEMS APPLICATIONS
      H.4.2 Types of Systems
          Subjects: Decision support (e.g., MIS)

  K. Computing Milieux
  K.4 COMPUTERS AND SOCIETY
      K.4.0 General
  K.6 MANAGEMENT OF COMPUTING AND INFORMATION SYSTEMS
      K.6.2 Installation Management

General Terms:
Design, Management

Keywords:
complexity metrics, configuration complexity, decision complexity