A validation methodology for agent-based simulations

ABSTRACT

Validity forms the basic prerequisite for every simulation model, therefore also for reasonable usage of the agent-based simulation paradigm. However, models based on the multi-agent system metaphor tend to need some particular approaches. In this paper, I propose a process for validating agent-based simulation models that combines face validation, sensitivity analysis, calibration and statistical validation.

REFERENCES

• 1: R. Axtell, R. Axelrod, J. E. Cohen, and M. D. Cohen. Aligning simulation models - a case study and results. Computational and Mathematical Organization Theory, 1:123--141, 1996.
• 2: O. Balci. Validation, verification and testing techniques troughout the life cycle of a simulation study. Annals of Operations Research, 53:121--173, 1994.
• 3: O. Barreteau et al. Our companion modeling approach. Journal of Artificial Societies and Social Simulation, 6(2), 2003.
• 4: K. Carley. Validating computational models. Technical report, Carnegie Mellon University, 1996.
• 5: J. Doran and M. Palmer. The eos project: Integrating two models of palaeolithic social change. In N. Gilbert and R. Conte, editors, Artificial Societies - The Computer Simulation of Social Life, chapter 6, pages 103--125. UCL, 1995.
• 6: Joshua M. Epstein, Generative Social Science: Studies in Agent-Based Computational Modeling (Princeton Studies in Complexity), Princeton University Press, Princeton, NJ, 2007
• 7: M. Fehler, F. Klügl, and F. Puppe. Learning for analysis and calibration in agent-based simulation. In Proc. of the ALAMAS 2006, Bruessels, 2006.
• 8: P. Guyot and S. Honiden. Agent-based participatory simulations: Merging multi-agent systems and role-playing games. Journal of Artificial Societies and Social Simulation, 9(4), 2006.
• 9: L. R. Izquierdo and J. G. Polhill. Is your model susceptible to floating-point errors? Journal of Artificial Societies and Social Simulation, 9(4):4, 2006.
• 10: Averill M. Law, How to build valid and credible simulation models, Proceedings of the 37th conference on Winter simulation, December 04-07, 2005, Orlando, Florida
• 11: Averill M. Law , W. David Kelton, Simulation Modeling and Analysis, McGraw-Hill Higher Education, 1997
• 12: R. Leombruni, M. Richiardi, N. J. Saam, and M. Sonnessa. A common protocol for agent-based social simulation. Journal of Artificial Societies and Social Simulation, 9(1), 2006.
• 13: S. Moss, T. E. Downing, and J. Rouchier. Demonstrating the role of stakeholder participation: An agent based social simulation model of water demand policy and response. Technical Report CPM Report No.: 00--76, CPM, 2000.
• 14: R. Suleiman, K. G. Troitzsch, and N. Gilbert, editors. Tools and Techniques for Social Science Simulation. Physica-Verlag, 2000.
• 15: K. G. Troitzsch. Validating simulation models. In Proceedings of the 18th European Simulation Multiconference, SCS Europe, 2004.
• 16: Bernard P. Zeigler, Theory of Modelling and Simulation, Krieger Publishing Co., Inc., Melbourne, FL, 1984

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.4 Software/Program Verification
          Subjects: Validation

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.4 Software/Program Verification
          Subjects: Statistical methods
      D.2.5 Testing and Debugging

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.11 Distributed Artificial Intelligence
            Subjects: Intelligent agents
  I.6 SIMULATION AND MODELING
      I.6.4 Model Validation and Analysis

General Terms:
Design, Management, Verification