Machine Learning Perspectives of Agent-Based Models

Inhaltsverzeichnis

1. Frontmatter

2. Chapter 1. Introduction

   Pedro Campos

   Abstract

   We reside in an era marked by continual crises that directly affect the real economy. The recent COVID-19 pandemic represented one of the most significant challenges since World War II. Financial turmoil and continuous conflicts highlight the world’s instability, prompting questions like: how do we handle such widespread uncertainty? Can we still make forecasts amidst this instability?”

3. Agent-Based Modelling and Machine Learning

   1. Frontmatter

   2. Chapter 2. Agent-Based Modeling and Learning in Economics: An Overview

      Pedro Campos, Anand Rao, Pavel Brazdil

      Abstract

      The fields of Agent-Based Modelling (ABM) and Multi-Agent Systems (MAS) have attracted the attention of many researchers over the past two or three decades, as they model the way our society is organized. So, developing such systems could help us to advance not only the general understanding, but also be used to help to guide decisions. In addition, crises have followed one another, from financial crises to pandemics and wars, with significant economic and social impacts. So, a question arises how these ABM e MAS can be developed. Using a manual approach does not sound right in the twenty-first century, when machine learning invaded virtually all areas of science. In this chapter we reflect on the role of learning, or in particular, machine learning, in multi-agent systems (MAS) and also the effects of crisis.

4. Agent-Based Models in the Context of COVID19

   1. Frontmatter

   2. Chapter 3. Epidemiology Modelling

      Arit Kumar Bishwas, Anand Rao

      Abstract

      The classic way of modeling the progression of an infectious disease has been called the SIR (Susceptible-Infected-Recovered) model. Further refinements of this types of models capture additional states like the SEIRD model (Susceptible, Exposed, Infected, Recovered, and Dead) that captures the exposure and death states. The COVID-19 pandemic has resulted in a number of these models—built for different countries—that capture additional states. We examine current literature in this rich area of epidemiological models including states for contact, quarantined, not quarantined, pre-symptomatic and pre-asymptomatic, symptomatic and asymptomatic states, hospitalization, and immunized. The more states of infection a model captures, the more it facilitates fine-grained decision-making. We review these new models and how they have been used during the pandemic to make spatio-temporal predictions on the progression of COVID19

   3. Chapter 4. Agent-Based Behavioral Models: Modeling COVID19 Behavior

      Anand Rao, Arit Kumar Bishwas

      Abstract

      The dynamics of the COVID19 pandemic was largely dictated by the behavior of the people to the disease and the government interventions that were imposed to control the disease. In this chapter we use agent-based modeling to model the behavior of individuals, including their mobility, social distancing, propensity to wear masks, pandemic fear, pandemic fatigue, and a number of other behaviors. We will also examine the interplay between these behaviors and the different government restrictions, such as, stay-at-home order, bar closure, restaurant closure, ban on large gatherings, etc.

   4. Chapter 5. COVID-19 Epidemiological, Behavioral, and Economic Model

      Anand Rao, Sindy Ma, Mark Paich, Joseph Voyles

      Abstract

      COVID19 had a significant impact not only on the health of individuals across the globe, it also impacted their financial well-being. In addition, COVID19 also had a major impact on companies and their operations, governments and also the larger global macro-economic environment. In this article, we outline how we built a sophisticated collection of agent-based models that addressed the disease progression, the behavioral change of citizens, and the economic impact across different industry sectors, including hospital networks, health insurers, pharmaceutical and life sciences, and retail. Although the focus of these models were US the general principles apply across the globe. We use this as a case study to illustrate how to scope agent-based models, build them, calibrate them, and continuously refine them. We also use this case study to highlight the importance of designing agent-based models in a modular fashion to enable the linking of multiple ABMs.

5. Creating Agent-Based Models of Crisis in Python, and R

   1. Frontmatter

   2. Chapter 6. MyWealth: A Simple Model of Economic Exchange in Python

      Joaquim Margarido, Pedro Campos

      Abstract

      The increasing focus on Agent-Based Models (ABMs) in economic studies results from the inadequacy of prevailing theoretical frameworks, notably exposed during the 2007–2008 global financial crisis and the COVID-19 pandemic. Complex models like EURACE demand extensive validation, often facing overfitting challenges. To facilitate understanding, simple games serve as effective tools for introducing agent-based modeling to beginners. Schelling’s model of social segretation for example, illustrates emergence, emphasizing contingent behavior influenced by individual goals and purposes. This chapter introduces MyWealth, a basic economic exchange model with a learning component, akin to the Simple Economy model. We create this model from scratch in Python, simulating simplified economies, aiding in teaching economic concepts and agent-based modeling principles.

   3. Chapter 7. The Ultimatum Game as a Paradigm for Learning Agents: A Python Adventure

      Joaquim Margarido, Pedro Campos

      Abstract

      This chapter explores the application of Agent-Based Models (ABMs) in understanding economic crises and pandemics, employing a machine learning approach for description and prediction. Emphasizing Game Theory’s importance in ABMs, the Ultimatum game serves as a paradigm for learning agents. The game involves a proposer and responder, deciding on a split, with consequences based on acceptance or refusal. The implementation, conducted step by step in Python, initially establishes a baseline model with proposers using fair or unfair split strategies. Subsequently, two learning strategies, Fictitious Play and Reinforcement Learning, are introduced. Fictitious Play minimizes responder rejections, while Reinforcement Learning optimizes action policies through sequential decision processes. In the model, a crisis is triggered halfway through the defined iterations, that entails responders increasing their acceptance threshold by 50%, demonstrating the synergy between ABMs, Game Theory, and machine learning in economic modeling.

   4. Chapter 8. Alternative Machine Learning Approaches for an Agent-Based Model of the Ultimatum Game Using R

      Pedro Campos, José Matos, Joaquim Margarido

      Abstract

      We provide a straightforward demonstration of the potential of different types of agent-based learning in the context of the Ultimatum Game. A recurring iteration of the Ultimatum Game is explored through Agent-Based Models (ABM), in which agents—representing the players—engage in repeated interactions following predefined rules. Leveraging the capabilities of Machine Learning we aim to harness the agents’ capacity to acquire strategies for optimizing their earnings within this game. Illustrative simplified examples of Fictitious Play, Reinforcement Learning, and Classifier systems are developed in R. The Classifier systems are based on Decision Trees that enable agents to learn from previous interactions and use background knowledge. The prior knowledge is based on the previous results of Reinforcement Learning in two different ways: on one hand, agent decisions are random (emphasizing exploration), and on the other hand, agent decisions are “epsilon-greedy”, emphasizing exploitation. We compare the gains of the agents in the different setups. We then assume that agents are placed in networks to develop a more sophisticated setup and explore the possibilities of Transfer Learning among agents, where some teach others how to learn.

6. Case Studies: Agent-Based Learning and Crisis Using R, Netlogo, and Julia

   1. Frontmatter

   2. Chapter 9. An Agent-Based Epidemic Modeling in Julia

      Ali R. Vahdati

      Abstract

      Agent-based models (ABMs) are computational simulations with autonomous agents interacting within an environment, offering a distinct scientific approach alongside inductive and deductive inference. In contrast to inductive inference, which identifies patterns from data, and deductive inference, which tests the consequences of assumptions, ABMs blend these methods. They commence with assumptions akin to deduction but don’t generate theorems. Instead, ABMs produce simulated data based on a set of rules, allowing inductive analysis. Especially valuable for complex systems, ABMs excel in modeling non-aggregate, emergent behaviors not derived from basic interactions.

      ABMs present advantages over other models by being more realistic, accommodating complex relationships and heterogeneous populations in diverse environments. Their flexibility allows integration of various aspects, from social to environmental factors. Originating in the 1970s, large-scale ABMs gained prominence in the 1990s with increased computing power. They find applications in diverse domains, such as human migrations, climate change effects on civilizations, disease propagation, plant root colonization, cell alterations’ impact on multicellular behavior, economics, and transport modeling.

      Despite their potential, ABMs face challenges like justifying model inputs and dealing with large, complex outputs. This chapter advocates for Julia language and the Agents.jl framework for ABM due to their efficiency in model construction, execution, and analysis, emphasizing the importance of suitable tools for gaining knowledge effectively from ABMs.

   3. Chapter 10. Portfolio Management and Crises: A Multi-Armed Bandit Approach

      Inês Ferreira, Marta Moraes

      Abstract

      In this chapter we develop and implement a Multi-Armed Bandit (MAB) to optimize equity portfolios. Then, we analyse the impact that a crisis can have on the system. The implementation of both the MAB algorithm and the crisis is made using R and RStudio software.

   4. Chapter 11. Organizational Learning from Crises with Machine Learning and Agent-Based Models

      Friederike Wall, Pedro Campos

      Abstract

      Organizational learning from crisis reflects that organizations may learn from their own and other’s experiences to increase organizational resilience and preparedness. However, it is for long been noticed that organizational learning from crisis often is hampered by barriers like, for example, cognitive narrowing and fixation or rigidity in core beliefs and assumptions—intermingled with actors’ building up of “defense lines” against mutual attributions of failure. Against this background, this chapter focuses on how machine learning and agent-based models could contribute to organizational learning from crisis. In particular, adopting the accountability perspective of organizational learning and building on Senge’s The fifth discipline: The art and practice of the learning organization. Doubleday, New York, 1990 prominent “five disciplines”, we derive a framework for machine learning and agent-based models to facilitate organizational learning. Moreover, we propose “systemic learning” as an integrated learning form that may capture the five disciplines for organizational learning.

   5. Chapter 12. Strategic Alliances in NetLogo: A Flocking Algorithm with Reinforcement Learning

      Sónia Teixeira, Pedro Campos

      Abstract

      The evolution of markets provides a change in the way organisations act. To improve their competitive performance and stay on the market, organisations often adopt a strategy to establish agreements with other organisations, known as strategic alliances. Several tools, algorithms, and computational systems call upon other sciences as a source of inspiration. In this work we explore flocking behaviour, a paradigm of biology, to analyse the collective intelligence behaviour that emerges from a group of individuals or firms. Inspired by the Cucker and Smale algorithm (C-S), we propose a new version of the flocking algorithm, AllFlock, applied to strategic alliances, considering a learning mechanism. For this new approach, metrics were obtained for the parameters of the C-S algorithm: position, velocity, and influence. The latter uses cooperative games, adapted mechanisms, and methods currently explored in reinforcement learning. We have used Netlogo as the modelling environment. Five parameter configurations were analysed. For each of those configurations, the average number of iterations, the permanence rate of organisations in the alliance, and the average growth of the organisations were computed. The behaviour of the organisations reveals a tendency for convergence, confirming the existence of flocking behaviour.

   6. Chapter 13. Exploring the Efficiency vs. Fairness Behavioural Spectrum in Multi-Agent Deep Reinforcement Learning

      Margarida Silva, Zafeiris Kokkinogenis, Jeremy Pitt, Rosaldo J. F. Rossetti

      Abstract

      The concept of fairness has been studied in philosophy and economics for thousands of years, so human actors in social systems have had plenty of time to “learn” what does, and does not, work. Yet, only recently. However, it is a relatively new question how software agents in a multi-agent system can use Reinforcement Learning models to develop an architecture that promotes equality or equity in the distribution of rewards to the agents within the system. Recent significant contributions have focused on optimising for efficiency based on the assumption that efficiency and fairness are opposites to be traded off against each other, but actually, the result of mixing fair and efficient policies is unknown in multi-agent reinforcement learning settings. In this work, we experiment with fair and efficient behaviours jointly, based on an extension of the state-of-the-art model in fairness SOTO that intertwines efficient and equitable recommendations. We analyse the fair versus efficient behavioural spectrum in the Matthew Effect and Traffic Light Control problems, finding some solutions that outperform the baseline SOTO and others that outperform a selfish baseline with comparable architectural design. We conclude it is possible to optimise for fairness and efficiency and this is important when computation of the reward distribution has to be paid for from the rewards themselves.

   7. Chapter 14. Resilient Agent-Based Networks in the Automotive Industry

      Ana Nogueira, Conceição Rocha, Pedro Campos

      Abstract

      The present work is inspired by the aftermarket companies of the automotive industry. The goal is to investigate how companies react to market change, by understanding the effect of a perturbation (such as a business cessation) on the rest of the companies that are interconnected through peer-to-peer relationships. An agent-based model has been developed that simulates a multilayer network involving different types of companies: suppliers, aftermarket companies; retailers and consumers. The effect of the cessation is measured by the resilience of the multilayer network after suffering the perturbation. The multilayer network is inspired in a business model of the automobile industry’s aftermarket and each type of company has some defined characteristics. The agent-based model produces the network dynamics due to the changes in its configuration throughout time. No learning mechanism is introduced in this work. We demonstrate that the number of links, the volume of sales and the total profit of a node in the network has an impact on its survival throughout time.