A validation methodology for agent-based simulations

  title={A validation methodology for agent-based simulations},
  author={F. Kluegl},
  • F. Kluegl
  • Published in SAC 2008
  • Computer Science
Validity forms the basic prerequisite for every simulation model, therefore also for reasonable usage of the agent-based simulation paradigm. However, models based on the multiagent system metaphor tend to need some particular approaches. In this paper, I propose a process for validating agent-based simulation models that combines face validation, sensitivity analysis, calibration and statistical validation. 

Figures and Tables from this paper

A generic testing framework for agent-based simulation models

This paper designed and developed a generic testing framework for agent-based simulation models to conduct validation and verification of models and demonstrates its effectiveness by showing its applicability on a realistic agent- based simulation case study.

Agent-Based Modeling and Simulation

This article gives an introduction to agent-based modeling and simulation (ABMS), focusing on its generative and bottom-up nature, its advantages as well as its pitfalls, and selected tools and well-known applications.

Agent-based simulation validation: A case study in demographic simulation

The results of a white-box validation performed in an agent-based simulator for population dynamics, which provides a way to simulate the demographic evolution of large populations in a parallel environment is presented.

To Calibrate & Validate an Agent-based Simulation Model - An Application of the Combination Framework of BI Solution & Multi-agent Platform

This paper proposes a calibration and validation approach for an agent-based model, using a Combination Framework of Business intelligence solution and Multi-agent platform (CFBM), which is a logical framework dedicated to the management of the input and output data in simulations, as well as the corresponding empirical datasets in an integrated way.


A structured way of documenting agent-based simulation models is presented by presenting a documentation framework that consists of six different categories of model information: metadata, informal model characterization, model contents, expected simulation behavior, experimental frame, and passed tests.

Towards Pattern-Oriented Design of Agent-Based Simulation Models

This work proposes to address the problem by providing a set of model design patterns inspired by patterns in Software Engineering for capturing the reusable essence of a solution to specific partial modeling problem.

State space analysis for model plausibility validation in multi-agent system simulation of urban policies

Some of the difficulties in establishing the verification and validation of agent-based models (ABMs) are considered and the use of coloured Petri net (CPN) formalism to specify agent behaviour to check whether the model looks and behaves logically is proposed.

From Formal Modelling to Agent Simulation Execution and Testing

This work presents an approach to agent-based simulation development using formal modelling, i.e. stream X-Machines, that combines the power of executable specifications and test case generation.

Validation of Agent-Based Simulation through Human Computation: An Example of Crowd Simulation

This paper proposes a new technique for validation of agent-based models, particularly those which relate to human behavior which adopts ideas from the field of Human Computation as a means of collecting large amounts of contextual behavioral data.



Learning for Analysis and Calibration in Agent-Based Simulation

The use of agent learning techniques are explored as a means for analyzing the effect of particular agent decision model structures on simulation model validity for evaluating its limits according to the available data.

Agent-Based Participatory Simulations: Merging Multi-Agent Systems and Role-Playing Games

It is argued that agent-based participatory simulations are also a significant improvement over the MAS/RPG approach, opening new perspectives and solving some of the problems generated by the joint use of role-playing games and multi-agent systems.

How to build valid and credible simulation models

  • A. Law
  • Computer Science
    2008 Winter Simulation Conference
  • 2008
In this tutorial, techniques for building valid and credible simulation models are presented and the importance of a definitive problem formulation, discussions with subject-matter experts, and interacting with the decision-maker on a regular basis are discussed.

Artificial Societies: The Computer Simulation of Social Life

An exploration of the implications of developments in artificial intelligence for social scientific research, which builds on the theoretical and methodological insights provided by "Simulating

Simulation Modeling and Analysis

The text is designed for a one-term or two-quarter course in simulation offered in departments of industrial engineering, business, computer science and operations research.


This paper discusses aspects of validating simulation models designed to describe, explain and predict realworld phenomena. It starts with a short review of arguments used in the simsoc mailing list

Is Your Model Susceptible to Floating-Point Errors?

A framework that highlights the features of computer models that make them especially vulnerable to floating-point errors, and suggests ways in which the impact of such errors can be mitigated, and is illustrated by applying it to six example agent-based models in the literature.

Aligning simulation models: A case study and results

This paper develops the concepts and methods of a process we will call “alignment of computational models” or “docking” for short. Alignment is needed to determine whether two models can produce the

Validation, verification, and testing techniques throughout the life cycle of a simulation study

  • O. Balci
  • Engineering
    Proceedings of Winter Simulation Conference
  • 1994
Current software VV&T techniques and current simulation model VV &T techniques are surveyed and how they can all be applied throughout the life cycle of a simulation study are described.

Generative social science: Studies in agent-based computational modeling

  • M. Batty
  • Sociology, Political Science
  • 2008