Skip to main content
SpringerLink
Log in

Checking Simulations: Detecting and Avoiding Errors and Artefacts

  • Chapter
  • First Online:
Simulating Social Complexity

Part of the book series: Understanding Complex Systems ((UCS))

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    By input parameters in this statement we mean “everything that may affect the output of the model”, e.g. the random seed, the pseudo-random number generator employed and, potentially, information about the microprocessor and operating system on which the simulation was run, if these could make a difference.

  2. 2.

    The reader can see an interesting comparative analysis between agent-based and equation-based modelling in (Parunak et al. 1998).

  3. 3.

    Note that the thematician faces a similar problem when building his non-formal model. There are potentially an infinite number of models for one single target system.

  4. 4.

    Each individual member of this set can be understood as a different model or, alternatively, as a different parameterisation of one single –more general– model that would itself define the whole set.

  5. 5.

    There are some interesting attempts with INGENIAS (Pavón and Gómez-Sanz 2003) to use modelling and visual languages as programming languages rather than merely as design languages (Sansores and Pavón 2005; Sansores et al. 2006. These efforts are aimed at automatically generating several implementations of one single executable model (in various different simulation platforms).

  6. 6.

    See a complete epistemic review of the validation problem in Kleindorfer et al. (1998).

References

  • Axelrod RM (1997a) Advancing the art of simulation in the social sciences. In: Conte R, Hegselmann R, Terna P (eds) Simulating social phenomena, vol 456, Lecture notes in economics and mathematical systems. Springer, Berlin, pp 21–40

    Google Scholar 

  • Axelrod RM (1997b) The dissemination of culture: a model with local convergence and global polarization. J Confl Resolut 41(2):203–226

    Article  Google Scholar 

  • Axtell RL (2000) Why agents? On the varied motivations for agent computing in the social sciences. In: Macal CM, Sallach D (eds) Proceedings of the workshop on agent simulation: applications, models, and tools. Argonne National Laboratory, Argonne, pp 3–24

    Google Scholar 

  • Axtell RL, Epstein JM (1994) Agent based modeling: understanding our creations. The Bulletin of the Santa Fe Institute, Winter, pp 28–32

    Google Scholar 

  • Bigbee T, Cioffi-Revilla C, Luke S (2007) Replication of sugarscape using MASON. In: Terano T, Kita H, Deguchi H, Kijima K (eds) Agent-based approaches in economic and social complex systems IV: post-proceedings of the AESCS international workshop 2005. Springer, Tokyo, pp 183–190

    Google Scholar 

  • Bonabeau E (2002) Agent-based modeling: methods and techniques for simulating human systems. Proc Natl Acad Sci U S A 99(2):7280–7287

    Article  Google Scholar 

  • Castellano C, Marsili M, Vespignani A (2000) Nonequilibrium phase transition in a model for social influence. Phys Rev Lett 85(16):3536–3539

    Article  Google Scholar 

  • Christley S, Xiang X, Madey G (2004) Ontology for agent-based modeling and simulation. In: Macal CM, Sallach D, North MJ (eds) Proceedings of the agent 2004 conference on social dynamics: interaction, reflexivity and emergence. Argonne National laboratory/The University of Chicago, Chicago. http://www.agent2005.anl.gov/Agent2004.pdf

  • Cioffi-Revilla C (2002) Invariance and universality in social agent-based simulations. Proc Natl Acad Sci U S A 99(3):7314–7316

    Article  Google Scholar 

  • Conlisk J (1996) Why bounded rationality? J Econ Lit 34(2):669–700

    Google Scholar 

  • David N (2013) Validating simulations. Chapter 8 in this volume

    Google Scholar 

  • Drogoul A, Vanbergue D, Meurisse T (2003) Multi-agent based simulation: where are the agents? In: Sichman JS, Bousquet F, Davidsson P (eds) Proceedings of MABS 2002 multi-agent-based simulation, vol 2581, Lecture notes in computer science. Springer, Bologna, pp 1–15

    Google Scholar 

  • Edmonds B (2001) The use of models: making MABS actually work. In: Moss S, Davidsson P (eds) Multi-agent-based simulation, vol 1979, Lecture notes in artificial intelligence. Springer, Berlin, pp 15–32

    Chapter  Google Scholar 

  • Edmonds B (2005) Simulation and complexity: how they can relate. In: Feldmann V, Mühlfeld K (eds) Virtual worlds of precision: computer-based simulations in the sciences and social sciences. Lit-Verlag, Münster, pp 5–32

    Google Scholar 

  • Edmonds B, Hales D (2003) Replication, replication and replication: some hard lessons from model alignment. J Artif Soc Soc Simulat 6(4). http://jasss.soc.surrey.ac.uk/6/4/11.html

  • Edmonds B, Hales D (2005) Computational simulation as theoretical experiment. J Math Sociol 29:1–24

    Article  Google Scholar 

  • Edwards M, Huet S, Goreaud F, Deffuant G (2003) Comparing an individual-based model of behaviour diffusion with its mean field aggregate approximation. J Artif Soc Soc Simulat 6(4). http://jasss.soc.surrey.ac.uk/6/4/9.html

  • Epstein JM (1999) Agent-based computational models and generative social science. Complexity 4(5):41–60

    Article  MathSciNet  Google Scholar 

  • Epstein JM (2008) Why model? J Artif Soc Soc Simul 11(4). http://jasss.soc.surrey.ac.uk/11/4/12.html

  • Epstein JM, Axtell RL (1996) Growing artificial societies: social science from the bottom up. Brookings Institution Press/MIT Press, Cambridge, MA

    Google Scholar 

  • Fensel D (2001) Ontologies: a silver bullet for knowledge management and electronic commerce. Springer, Berlin

    MATH  Google Scholar 

  • Galán JM, Izquierdo LR (2005) Appearances can be deceiving: lessons learned re-implementing Axelrod’s ‘Evolutionary approach to norms’. J Artif Soc Soc Simulat 8(3). http://jasss.soc.surrey.ac.uk/8/3/2.html

  • Galán JM et al (2009) Errors and artefacts in agent-based modelling. J Artif Soc Soc Simulat 12(1). http://jasss.soc.surrey.ac.uk/12/1/1.html

  • Gilbert N (1999) Simulation: a new way of doing social science. Am Behav Sci 42(10):1485–1487

    Google Scholar 

  • Gilbert N (2007) Agent-based models. Sage, London

    Google Scholar 

  • Gilbert N, Terna P (2000) How to build and use agent-based models in social science. Mind Soc 1(1):57–72

    Article  Google Scholar 

  • Gilbert N, Troitzsch KG (1999) Simulation for the social scientist. Open University Press, Buckingham

    Google Scholar 

  • Gotts NM, Polhill JG, Adam WJ (2003) Simulation and analysis in agent-based modelling of land use change. In: Online proceedings of the first conference of the European Social Simulation Association, Groningen, 18–21 Sept 2003. http://www.uni-koblenz.de/~essa/ESSA2003/gotts_polhill_adam-rev.pdf

  • Gruber TR (1993) A translation approach to portable ontology specifications. Knowl Acquis 5(2):199–220

    Article  Google Scholar 

  • Hare M, Deadman P (2004) Further towards a taxonomy of agent-based simulation models in environmental management. Math Comput Simulat 64(1):25–40

    Article  MathSciNet  MATH  Google Scholar 

  • Hernández C (2004) Herbert A Simon, 1916–2001, y el Futuro de la Ciencia Económica. Revista Europea De Dirección y Economía De La Empresa 13(2):7–23

    Google Scholar 

  • Heywood JG, Masuda K, Rautmann R, Solonnikov VA (eds) (1990) The Navier–Stokes equations: theory and numerical methods. In: Proceedings of a conference held at Oberwolfach, FRG, 18–24, Sept 1988 (Lecture notes in mathematics), vol 1431. Springer, Berlin

    Google Scholar 

  • Holland JH, Miller JH (1991) Artificial adaptive agents in economic theory. Am Econ Rev 81(2):365–370

    Google Scholar 

  • Izquierdo LR, Polhill JG (2006) Is your model susceptible to floating point errors? J Artif Soc Soc Simulat 9(4). http://jasss.soc.surrey.ac.uk/9/4/4.html

  • Kleijnen JPC (1995) Verification and validation of simulation models. Eur J Oper Res 82(1):145–162

    Article  MATH  Google Scholar 

  • Kleindorfer GB, O’Neill L, Ganeshan R (1998) Validation in simulation: various positions in the philosophy of science. Manage Sci 44(8):1087–1099

    Article  MATH  Google Scholar 

  • Klemm K, Eguíluz V, Toral R, San Miguel M (2003a) Role of dimensionality in Axelrod’s model for the dissemination of culture. Phys A 327:1–5

    Article  MathSciNet  MATH  Google Scholar 

  • Klemm K, Eguíluz V, Toral R, San Miguel M (2003b) Global culture: a noise-induced transition in finite systems. Phys Rev E 67(4):045101

    Article  Google Scholar 

  • Klemm K, Eguíluz V, Toral R, San Miguel M (2003c) Nonequilibrium transitions in complex networks: a model of social interaction. Phys Rev E 67(2):026120

    Article  Google Scholar 

  • Klemm K, Eguíluz V, Toral R, San Miguel M (2005) Globalization, polarization and cultural drift. J Econ Dyn Control 29(1–2):321–334

    Article  MATH  Google Scholar 

  • Kluver J, Stoica C (2003) Simulations of group dynamics with different models. J Artif Soc Soc Simulat 6(4). http://jasss.soc.surrey.ac.uk/6/4/8.html

  • Leombruni R, Richiardi M (2005) Why are economists sceptical about agent-based simulations? Phys A 355:103–109

    Article  MathSciNet  Google Scholar 

  • Moss S (2001) Game theory: limitations and an alternative. J Artif Soc Soc Simulat 4(2). http://jasss.soc.surrey.ac.uk/4/2/2.html

  • Moss S (2002) Agent based modelling for integrated assessment. Integr Assess 3(1):63–77

    Article  Google Scholar 

  • Moss S, Edmonds B, Wallis S (1997) Validation and verification of computational models with multiple cognitive agents (Report no. 97–25). Centre for Policy Modelling, Manchester. http://cfpm.org/cpmrep25.html

  • Ostrom T (1988) Computer simulation: the third symbol system. J Exp Soc Psychol 24(5):381–392

    Article  Google Scholar 

  • Parunak HVD, Savit R, Riolo RL (1998) Agent-based modeling vs. equation-based modeling: a case study and users’ guide. In: Sichman JS, Conte R, Gilbert N (eds) Multi-agent systems and agent-based simulation, vol 1534, Lecture notes in artificial intelligence. Springer, Berlin, pp 10–25

    Chapter  Google Scholar 

  • Pavón J, Gómez-Sanz J (2003) Agent oriented software engineering with INGENIAS. In: Marik V, Müller J, Pechoucek M (eds) Multi-agent systems and applications III, 3rd international central and eastern European conference on multi-agent systems, CEEMAS 2003 (Lecture notes in artificial intelligence), vol 2691. Springer, Berlin, pp 394–403

    Google Scholar 

  • Pignotti E, Edwards P, Preece A, Polhill JG, Gotts NM (2005) Semantic support for computational land-use modelling. In: Proceedings of the 5th international symposium on cluster computing and the grid (CCGRID 2005). IEEE Press, Piscataway, pp 840–847

    Google Scholar 

  • Polhill JG, Gotts NM (2006) A new approach to modelling frameworks. In: Proceedings of the first world congress on social simulation, vol 1. Kyoto, 21–25 Aug 2006, pp 215–222

    Google Scholar 

  • Polhill JG, Izquierdo LR (2005) Lessons learned from converting the artificial stock market to interval arithmetic. J Artif Soc Soc Simulat 8(2). http://jasss.soc.surrey.ac.uk/8/2/2.html

  • Polhill JG, Izquierdo LR, Gotts NM (2005) The ghost in the model (and other effects of floating point arithmetic). J Artif Soc Soc Simulat 8(1). http://jasss.soc.surrey.ac.uk/8/1/5.html

  • Polhill JG, Izquierdo LR, Gotts NM (2006) What every agent based modeller should know about floating point arithmetic. Environ Model Software 21(3):283–309

    Article  Google Scholar 

  • Riolo RL, Cohen MD, Axelrod RM (2001) Evolution of cooperation without reciprocity. Nature 411:441–443

    Article  Google Scholar 

  • Sakoda JM (1971) The checkerboard model of social interaction. J Math Sociol 1(1):119–132

    Article  Google Scholar 

  • Salvi R (2002) The Navier–Stokes equation: theory and numerical methods, Lecture notes in pure and applied mathematics. Marcel Dekker, New York

    MATH  Google Scholar 

  • Sansores C, Pavón J (2005) Agent-based simulation replication: a model driven architecture approach. In: Gelbukh AF, de Albornoz A, Terashima-Marín H (eds) Proceedings of MICAI 2005: advances in artificial intelligence, 4th Mexican international conference on artificial intelligence, Monterrey, 14–18 Nov 2005. Lecture notes in computer science, vol 3789. Springer, Berlin, pp 244–253

    Google Scholar 

  • Sansores C, Pavón J, Gómez-Sanz J (2006) Visual modeling for complex agent-based simulation systems. In: Sichman JS, Antunes L (eds) Multi-agent-based simulation VI, international workshop, MABS 2005, Utrecht, 25 July 2005, Revised and invited papers. Lecture notes in computer science, vol 3891. Springer, Berlin, pp 174–189

    Google Scholar 

  • Sargent RG (2003) Verification and validation of simulation models. In: Chick S, Sánchez PJ, Ferrin D, Morrice DJ (eds) Proceedings of the 2003 winter simulation conference. IEEE, Piscataway, pp 37–48

    Google Scholar 

  • Schelling TC (1971) Dynamic models of segregation. J Math Sociol 1(2):47–186

    Article  Google Scholar 

  • Schelling TC (1978) Micromotives and macrobehavior. Norton, New York

    Google Scholar 

  • Schmeiser BW (2001) Some myths and common errors in simulation experiments. In: Peters BA, Smith JS, Medeiros DJ, Rohrer MW (eds) Proceedings of the winter simulation conference, vol 1. Arlington, 9–12 Dec 2001, pp 39–46

    Google Scholar 

  • Takadama K, Suematsu YL, Sugimoto N, Nawa NE, Shimohara K (2003) Cross-element validation in multiagent-based simulation: switching learning mechanisms in agents. J Artif Soc Soc Simulat 6(4). http://jasss.soc.surrey.ac.uk/6/4/6.html

  • Taylor AJ (1983) The verification of dynamic simulation models. J Oper Res Soc 34(3):233–242

    Google Scholar 

  • Xu J, Gao Y, Madey G (2003) A docking experiment: swarm and repast for social network modeling. In: seventh annual swarm researchers conference (SwarmFest 2003), 13–15 Apr 2003, Notre Dame

    Google Scholar 

  • Yilmaz L (2006) Validation and verification of social processes within agent-based computational organization models. Comput Math Organ Theory 12(4):283–312

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The authors have benefited from the financial support of the Spanish Ministry of Education and Science (projects DPI2004–06590, DPI2005–05676 and TIN2008–06464–C03–02), the Spanish Ministry for Science and Innovation (CSD2010–00034) within the framework of CONSOLIDER-INGENIO 2010 and of the JCyL (projects VA006B09, BU034A08 and GREX251–2009). We are also very grateful to Nick Gotts, Gary Polhill, Bruce Edmonds and Cesáreo Hernández for many discussions on the philosophy of modelling.

Author information

Authors and Affiliations

  1. Departamento de Ingeniería Civil, Universidad de Burgos, Burgos, E-09001, Spain

    José M. Galán, Luis R. Izquierdo, José I. Santos & Ricardo del Olmo

  2. Departamento de Ingeniería Civil, Universidad de Burgos, Valladolid, E-47011, Spain

    Segismundo S. Izquierdo & Adolfo López-Paredes

Authors
  1. José M. Galán
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luis R. Izquierdo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Segismundo S. Izquierdo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. José I. Santos
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ricardo del Olmo
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Adolfo López-Paredes
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to José M. Galán .

Editor information

Editors and Affiliations

  1. CPM - Centre for Policy Modelling, Manchester Metropolitan University, Manchester, M1 3GH, United Kingdom

    Bruce Edmonds

  2. , Center for Policy Modelling, Manchester Metropolitan University, Aytoun Street, Manchester, M1 3GH, United Kingdom

    Ruth Meyer

Further Reading

Further Reading

Gilbert (2007) provides an excellent basic introduction to agent based modelling. Chapter 3 summarizes the different stages involved in an agent-based modelling project, including verification and validation. The paper entitled “Some myths and common errors in simulation experiments” (Schmeiser 2001) discusses briefly some of the most common errors found in simulation from a probabilistic and statistical perspective. The approach is not focused specifically on agent based modelling but on simulation in general. Yilmaz (2006) presents an analysis of the life cycle of a simulation study and proposes a process-centric perspective for the validation and verification of agent-based computational organization models. Finally, Chap. 8 in this volume (David 2013) discusses validation in detail.

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Galán, J.M., Izquierdo, L.R., Izquierdo, S.S., Santos, J.I., del Olmo, R., López-Paredes, A. (2013). Checking Simulations: Detecting and Avoiding Errors and Artefacts. In: Edmonds, B., Meyer, R. (eds) Simulating Social Complexity. Understanding Complex Systems. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-93813-2_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-93813-2_6

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-93812-5

  • Online ISBN: 978-3-540-93813-2

  • eBook Packages: Physics and AstronomyPhysics and Astronomy (R0)

Publish with us

Policies and ethics

Search

Navigation