Skip to main content
main-content
Top

About this book

This book draws inspiration from natural shepherding, whereby a farmer utilizes sheepdogs to herd sheep, to inspire a scalable and inherently human friendly approach to swarm control. The book discusses advanced artificial intelligence (AI) approaches needed to design smart robotic shepherding agents capable of controlling biological swarms or robotic swarms of unmanned vehicles. These smart shepherding agents are described with the techniques applicable to the control of Unmanned X Vehicles (UxVs) including air (unmanned aerial vehicles or UAVs), ground (unmanned ground vehicles or UGVs), underwater (unmanned underwater vehicles or UUVs), and on the surface of water (unmanned surface vehicles or USVs). This book proposes how smart ‘shepherds’ could be designed and used to guide a swarm of UxVs to achieve a goal while ameliorating typical communication bandwidth issues that arise in the control of multi agent systems. The book covers a wide range of topics ranging from the design of deep reinforcement learning models for shepherding a swarm, transparency in swarm guidance, and ontology-guided learning, to the design of smart swarm guidance methods for shepherding with UGVs and UAVs. The book extends the discussion to human-swarm teaming by looking into the real-time analysis of human data during human-swarm interaction, the concept of trust for human-swarm teaming, and the design of activity recognition systems for shepherding.

Presents a comprehensive look at human-swarm teaming;Tackles artificial intelligence techniques for swarm guidance;Provides artificial intelligence techniques for real-time human performance analysis.

Table of Contents

Frontmatter

Chapter 1. Smart Shepherding: Towards Transparent Artificial Intelligence Enabled Human-Swarm Teams

Abstract
The aim of this chapter is to uncover the beauty and complexity in the world of shepherding as we view it through the lens of Artificial Intelligence (AI) and Autonomous Systems (AS). In the pursuit of imitating human intelligence, AI researchers have made significant and vast contributions over decades. Yet even with such interest and activity from within industry and the academic community, general AI remains out of our reach. By comparison, this book aims for a less ambitious goal in trying to recreate the intelligence of a sheepdog. As our efforts display, even with this seemingly modest goal, there is a plethora of research opportunities where AI and AS still have a long way to go. Let us start this journey by asking the basic questions: what is shepherding and what makes shepherding an interesting problem? How does one design a smart shepherd for swarm guidance? What AI algorithms are required and how are they organised in a cognitive architecture to enable a smart shepherd? How does one design transparent AI for smart shepherding?
Hussein A. Abbass, Robert A. Hunjet

Shepherding Simulation

Frontmatter

Chapter 2. Shepherding Autonomous Goal-Focused Swarms in Unknown Environments Using Hilbert Space-Filling Paths

Abstract
A novel technique has been developed for autonomous swarm-based unknown environment scouting. A control method known as swarm shepherding was employed, which replicates the behaviour seen when a sheepdog guides a herd of sheep to an objective location. The guidance of the swarm agents was implemented using low computation cost, force-based behaviours. The exploration task was augmented by introducing swarm member role assignments, including a role which imposes a localised covering area for agents which stray too far from the swarm global centre of mass. The agents then proceeded to follow a Hilbert space-filling curve (HSFC) path within their localised region. The simulation results demonstrated that the inclusion of the HSFC paths improved the efficiency of goal-based exploration of the environment, which became more prominent with an increase in the density of the number of goals in the environment.
Nathan K. Long, Matthew Garratt, Karl Sammut, Daniel Sgarioto, Hussein A. Abbass

Chapter 3. Simulating Single and Multiple Sheepdogs Guidance of a Sheep Swarm

Abstract
Shepherding is a specific class of flocking behaviour where external agents (the shepherd) influence the movements of a group of agents (the flock). In nature, a powerful example is herding a flock of sheep by an influential sheepdog. When the sheepdog encroaches on the sheep’s influence zone, the sheep is essentially “repelled”. Optimising this phenomenon has many engineering applications, such as environmental protection, security, crowd, and agricultural control. In this chapter, we build on Strömbom et al’s adaptive switching algorithm to incorporate multiple sheepdog agents programmed with basic swarm intelligence rules. Our simulation results show that synergising shepherds with swarming behaviours improves the effectiveness of the shepherds as measured by the speed of collecting and driving the sheep towards a target destination while maintaining a more compact and cohesive flock.
Daniel Baxter, Matthew Garratt, Hussein A. Abbass

Chapter 4. The Influence of Stall Distance on Effective Shepherding of a Swarm

Abstract
In the shepherding problem, an external agent (the shepherd) attempts to influence the behaviour of a swarm of agents (the sheep) by steering them towards a goal that is known to the shepherd but not the sheep. This chapter outlines some of the dynamics inside a shepherding task known as herding, focusing on what we call the stall distance. We describe the stall distance as the minimum distance that the shepherd must maintain from all members of the swarm while carrying out the primary task of herding the swarm to a particular location. This chapter shows how herding performance is influenced by setting the value for stall distance to an appropriate level in a herding model. A connection is made between the concept of stall distance for shepherds herding sheep agents in the model and real-world shepherds herding a group of sheep.
Anthony Perry

Learning and Optimisation for Shepherding

Frontmatter

Chapter 5. Mission Planning for Shepherding a Swarm of Uninhabited Aerial Vehicles

Abstract
Uninhabited aerial vehicles (UAVs) are widely used in many areas for completing complex missions such as tracking targets, search and rescue, farming (shepherding in the traditional sense) and mapping. Mission planning for shepherding a UAV swarm is advantageous for Human-Swarm Teaming. While most research on shepherding see the shepherd as a simple reactive agent, a smart shepherd in a complex environment will need to consider many dimensions and sub-decisions to successfully guide a swarm through complex environment and towards a goal.
In this chapter, we review and offer formal definitions for the sub-problems required for a shepherd to complete a mission successfully. The swarm mission planning system needs to have decision modules capable of solving four main problems: task decomposition, task assignment, path planning and trajectory generation. These sub-problems are coupled differently depending on the scenario. This chapter defines these sub-problems in their general form and gives UAV swarm shepherding problem as a specific application. A brief review of the widely used algorithms for tackling these problems and the state of art of mission planning are also given in this chapter.
Jing Liu, Sreenatha Anavatti, Matthew Garratt, Hussein A. Abbass

Chapter 6. Towards Ontology-Guided Learning for Shepherding

Abstract
Shepherding offers an exciting application for machine learning research. Shepherding tasks are scalable in terms of both complexity and dimension. This scalability supports investigations into the generality of learned multi-agent solutions. Shepherding is also valuable for the study of how multi-agent learning systems transition from simulation to physical systems. This chapter reviews previous learning strategies for shepherding and highlights the advantages of applying prior knowledge to the design of learning systems for shepherding. It presents ontology guided learning, a hybrid learning approach to learning. Ontology guided learning will enable the application of symbolic prior knowledge to non-symbolic learning systems. This will allow a non-symbolic system to reason on abstract concepts, reduce dimensionality by partitioning the state and action space, increase transparency and allow learning to focus on the parametric rather than semantic parts of the problem, where it will likely be most effective. This chapter concludes by describing how ontology guided learning could be applied to the shepherding problem.
Benjamin Campbell

Chapter 7. Activity Recognition for Shepherding

Abstract
Activity recognition for shepherding is a way for an artificial intelligence system to learn and understand shepherding behaviours. The problem we describe is one of recognising behaviours within a shepherding environment, where a cognitive agent (the shepherd) influences agents within the system (sheep) through a shepherding actuator (sheepdog), to achieve an intent. Shepherding is pervasive in everyday life with AI agents, collections of animals, and humans all partaking in different forms. Activity recognition in this context is the generation of a transformation from sensor stream data to the perceived behaviour of an agent under observation from the perspective of an external observer. We present a method of classifying behaviour through the use of spatial data and codify action, behaviour, and intent states through a multi-level classification mapping process.
Adam J. Hepworth

Chapter 8. Stable Belief Estimation in Shepherd-Assisted Swarm Collective Decision Making

Abstract
Swarm collective decision making refers to the case where a swarm needs to make a decision based on different pieces of evidence collected by its individuals. This problem has been investigated by several recent studies which proposed strategies to enable the swarm to perform fast and accurate collective decision making. However, the performance of these strategies (in terms of its accuracy, speed and level of consensus) suffers significantly in complex environments. The aim of our work is to propose a collective decision-making strategy that promises a consistent performance across different levels of scenario complexity and achieves superiority over the existing strategies in highly complex scenarios. To achieve this aim, our proposed algorithm employs a shepherding agent to boost the performance of the swarm. The swarm members are only responsible for sensing the state of a feature distributed in the environment. Only the shepherd needs to be able to process position and navigation abilities to collect the swarm. The algorithm consists of two phases: exploration and belief sharing. In the exploration phase, swarm members navigate through an environment and sense its features. Then, in the belief sharing phase, a shepherding agent collects the swarm members together so that they can share their estimates and calculate their decisions. The results demonstrate that the proposed shepherding algorithm succeeds across different levels of scenario complexity. Additionally, the approach achieves high levels of accuracy and consensus in complex non-homogeneous environments where the baseline state-of-the-art algorithm fails.
Aya Hussein, Hussein A. Abbass

Sky Shepherding

Frontmatter

Chapter 9. Sky Shepherds: A Tale of a UAV and Sheep

Abstract
This chapter considers the evolution of a shepherding agent from the traditional sheep dog to a UAV capable of fostering the welfare of sheep throughout the shepherding task. Dorper sheep have been exposed to a single Sky Shepherd, with results of such screening tests discussed. Behaviours identified include an alert response, whereby flock members display curiosity toward the UAV. As testing progressed, flocks displayed synchronous alert and drive behaviour responses, with some sheep, hypothesised to be leader sheep, pausing periodically and displaying curiosity towards the UAV.
Kate J. Yaxley, Nathan McIntyre, Jayden Park, Jack Healey

Chapter 10. Apprenticeship Bootstrapping Reinforcement Learning for Sky Shepherding of a Ground Swarm in Gazebo

Abstract
The coordination of unmanned air–ground vehicles has been an active area due to the significant advantages of this coordination wherein unmanned air vehicles (UAVs) have a wide field of view, enabling them to effectively guide a swarm of unmanned ground vehicles (UGVs). Due to significant recent advances in artificial intelligence (AI), autonomous agents are being used to design more robust coordination of air–ground systems, reducing the intervention load of human operators and increasing the autonomy of unmanned air–ground systems. A guidance and control shepherding system design allows for single learning agent to influence and manage a larger swarm of rule-based entities. In this chapter, we present a learning algorithm for a sky shepherd-guiding rule-based AI-driven UGVs. The apprenticeship bootstrapping learning algorithm is introduced and is applied to the aerial shepherding task.
Hung Nguyen, Matthew Garratt, Hussein A. Abbass

Chapter 11. Logical Shepherd Assisting Air Traffic Controllers for Swarm UAV Traffic Control Systems

Abstract
In recent years, Unmanned Aerial Vehicles (UAVs) have attracted attentions from almost every industry. Their low cost, high accessibility, and low-risk compared to human-operated vehicles, created a unique opportunity for a variety of use cases in many application domains. The addition of these tele-operated, and sometimes autonomous, vehicles to the air traffic control environment imposed significant challenges and has been calling for appropriate UAV traffic control systems. The complexity of this situation increases manyfold when the UAVs need to work together as a swarm. Air traffic controllers are used to manage 20 or so aircraft separated according to strict guidelines. A highly dynamic, adaptive, fast, and large swarm of UAVs present unprecedented complexity. Shepherding offers an opportunity to provide a concept of a single sheepdog simultaneously guiding a large sheep flock. We present a logical shepherd that could act both in an autonomous mode or in a tele-operation mode by simply sitting in the hands of a swarm traffic controller. Due to the safety critical nature of the environment, we modified the concept of shepherding by designing an asynchronous shepherding algorithm coupled with a digital twin environment to assess consequences. Once the logical shepherd location and orientation is chosen by the human operator, the influence force vectors start to propagate asynchronously from one aircraft to another, maintaining separation assurance and safety constraints. The updated trajectory intent information of the UAVs gets displayed on the screen for the human operator to see if the change is acceptable or not. If acceptable, the recommendation is made and the UAVs commence to follow the new path.
Heba El-Fiqi, Kathryn Kasmarik, Hussein A. Abbass

Human-Shepherding Integration

Frontmatter

Chapter 12. Transparent Shepherding: A Rule-Based Learning Shepherd for Human Swarm Teaming

Abstract
This chapter aims to demonstrate how rule-based Artificial Intelligence algorithms can address a few human swarm teaming challenges. We will start from the challenges identified by the cognitive engineering community for building human autonomy teaming and how they scale to human swarm teaming. The discussion will follow with a description of rule-based machine learning with a focus on learning classifier systems as a representative for these algorithms and their benefits for human swarm teaming. Shepherding affords a human to manage a swarm by teaming with an autonomous single shepherd. A learning classifier system is designed to learn behaviour needed to be exhibited by the shepherd. Results demonstrate the effectiveness of the rule-based XCS model to capture shepherding behaviour, where the XCS model achieves comparable performance to the standard Strömbom’s shepherding method as measured by the number of steps needed by a sheep-dog to guide group of sheep to target destination. These results are promising and demonstrate that learning classifier systems could design autonomous shepherds for new type of shepherding tasks and scenarios that we may not have rules for today.
Essam Debie, Raul Fernandes Rojas, Justin Fidock, Michael Barlow, Kathryn Kasmarik, Sreenatha Anavatti, Matthew Garratt, Hussein A. Abbass

Chapter 13. Human Performance Operating Picture for Shepherding a Swarm of Autonomous Vehicles

Abstract
Due to many factors that range from ethical considerations and accountability to technological imperfection in autonomous systems, humans will continue to be an integral part of any meaningful autonomous system. While shepherding offers a technological concept that allows a human to operate a significantly larger number of autonomous systems that a human can handle in today’s environment, it is important to realise that a significant amount of accidents today are due to human error. The scalability promise that shepherding offers comes with possible challenges including those arising from the cognitive load imposed on human operators and the need to smoothly integrate the human, as a biological autonomous system, with the wider multi-agent autonomous system of future operating environments. In this chapter, we bring together the dimensions of this complex problem. We present carefully selected factors to cover human performance, especially for cognitively demanding tasks and situation awareness, and how these factors contribute to trust in the system. We then present the Human Factors Operating Picture (H-FOP), which offers a real-time situation awareness picture on human performance in this complex environment. We conclude with the concept of operation for integrating H-FOP with the human-swarm teaming problem, with a focus on the reliance of shepherding as the swarm guidance and control method.
Raul Fernandez Rojas, Essam Debie, Justin Fidock, Michael Barlow, Kathryn Kasmarik, Sreenatha Anavatti, Matthew Garratt, Hussein A. Abbass

Backmatter

Additional information

Premium Partner

    Image Credits