Skip to main content
main-content

Über dieses Buch

This book is open access under a CC BY 4.0 license.
This book establishes the foundations needed to realize the ultimate goals for artificial intelligence, such as autonomy and trustworthiness. Aimed at scientists, researchers, technologists, practitioners, and students, it brings together contributions offering the basics, the challenges and the state-of-the-art on trusted autonomous systems in a single volume. The book is structured in three parts, with chapters written by eminent researchers and outstanding practitioners and users in the field. The first part covers foundational artificial intelligence technologies, while the second part covers philosophical, practical and technological perspectives on trust. Lastly, the third part presents advanced topics necessary to create future trusted autonomous systems. The book augments theory with real-world applications including cyber security, defence and space.

Unsere Produktempfehlungen

Premium-Abo der Gesellschaft für Informatik

Sie erhalten uneingeschränkten Vollzugriff auf alle acht Fachgebiete von Springer Professional und damit auf über 45.000 Fachbücher und ca. 300 Fachzeitschriften.

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 69.000 Bücher
  • über 500 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Umwelt
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Testen Sie jetzt 30 Tage kostenlos.

Basis-Abo der Gesellschaft für Informatik

Sie erhalten uneingeschränkten Vollzugriff auf die Inhalte der Fachgebiete Business IT + Informatik und Management + Führung und damit auf über 30.000 Fachbücher und ca. 130 Fachzeitschriften.

Weitere Produktempfehlungen anzeigen

Inhaltsverzeichnis

Frontmatter

Open Access

Chapter 1. Foundations of Trusted Autonomy: An Introduction

Abstract
To aid in understanding the chapters to follow, a general conceptualisation of autonomy may be useful. Foundationally, autonomy is concerned with an agent that acts in an environment. However, this definition is insufficient for autonomy as it requires persistence (or resilience) to the hardships that the environment acts upon the agent. An agent whose first action ends in its demise would not demonstrate autonomy. The themes of autonomy then include agency, persistence and action.
Hussein A. Abbass, Jason Scholz, Darryn J. Reid

Autonomy

Frontmatter

Open Access

Chapter 2. Universal Artificial Intelligence

Practical Agents and Fundamental Challenges
Abstract
Foundational theories have contributed greatly to scientific progress in many fields. Examples include Zermelo-Fraenkel set theory in mathematics, and universal Turing machines in computer science. Universal Artificial Intelligence (UAI) is an increasingly well-studied foundational theory for artificial intelligence, based on ancient principles in the philosophy of science and modern developments in information and probability theory. Importantly, it refrains from making unrealistic Markov, ergodicity, or stationarity assumptions on the environment. UAI provides a theoretically optimal agent AIXI and principled ideas for constructing practical autonomous agents. The theory also makes it possible to establish formal results on the motivations of AI systems. Such results may greatly enhance the trustability of autonomous agents, and guide design choices towards more robust agent architectures and incentive schemes. Finally, UAI offers a deeper appreciation of fundamental problems such as the induction problem and the exploration-exploitation dilemma .
Tom Everitt, Marcus Hutter

Open Access

Chapter 3. Goal Reasoning and Trusted Autonomy

Abstract
This chapter discusses the topic of Goal Reasoning and its relation to Trusted Autonomy. Goal Reasoning studies how autonomous agents can extend their reasoning capabilities beyond their plans and actions, to consider their goals. Such capability allows a Goal Reasoning system to more intelligently react to unexpected events or changes in the environment. We present two models of Goal Reasoning: Goal-Driven Autonomy (GDA) and goal refinement. We then discuss several research topics related to each, and how they relate to the topic of Trusted Autonomy. Finally, we discuss several directions of ongoing work that are particularly interesting in the context of the chapter: using a model of inverse trust as a basis for adaptive autonomy, and studying how Goal Reasoning agents may choose to rebel (i.e., act contrary to a given command).
Benjamin Johnson, Michael W. Floyd, Alexandra Coman, Mark A. Wilson, David W. Aha

Open Access

Chapter 4. Social Planning for Trusted Autonomy

Abstract
In this chapter, we describe social planning mechanisms for constructing and representing explainable plans in human-agent interactions, addressing one aspect of what it will take to meet the requirements of a trusted autonomous system. Social planning is automated planning in which the planning agent maintains and reasons with an explicit model of the other agents, human or artificial, with which it interacts, including the humans’ goals, intentions, and beliefs, as well as their potential behaviours. The chapter includes a brief overview of the challenge of planning in human-agent teams, and an introduction to a recent body of technical work in multi-agent epistemic planning. The benefits of planning in the presence of nested belief reasoning and first-person multi-agent planning are illustrated in two scenarios, hence indicating how social planning could be used for planning human-agent interaction explicitly as part of an agent’s deliberation.
Tim Miller, Adrian R. Pearce, Liz Sonenberg

Open Access

Chapter 5. A Neuroevolutionary Approach to Adaptive Multi-agent Teams

Abstract
A multi-agent architecture called the Adaptive Team of Agents (ATA) is introduced, wherein homogeneous agents adopt specific roles in a team dynamically in order to address all the sub-tasks necessary to meet the team’s goals. Artificial neural networks are then trained by neuroevolution to produce an example of such a team, trained to solve the problem posed by a simple strategy game. The evolutionary algorithm is found to induce the necessary in situ adaptivity of behavior into the agents, even when controlled by stateless feed-forward networks.
Bobby D. Bryant, Risto Miikkulainen

Open Access

Chapter 6. The Blessing and Curse of Emergence in Swarm Intelligence Systems

Abstract
In an increasingly complex and interconnected world, there is an increasing need for autonomous systems that can control systems that are beyond the capabilities of human operators. One of the key issues to be addressed in developing trusted autonomous systems is dealing with the phenomenon of “emergence”; either by taking advantage of emergence or avoiding emergence. Swarm intelligence systems, based on the interaction of a large number of relatively simples agents, rely on emergent intelligence for their problem solving capabilities. When used in trusted autonomous systems, the emergent behaviour of swarm intelligence systems can be both a blessing and a curse.
John Harvey

Open Access

Chapter 7. Trusted Autonomous Game Play

Abstract
Game play by humans has always required autonomy and trust . Autonomy because a person chooses to play and takes in-game actions; and trust that the other players adopt a lusory attitude and abide by the rules of the game. The chapter highlights several areas of digital (computer) game design and development that will be revolutionised by the technology of, and framework of Trusted Autonomy: TA game AI that display emotional and other (non-logic) forms of intelligence; TA games that are aware of themselves and of the player, and self-modify to enhance play; TA communities that create a safe, fulfilling, and non-toxic environment; and TA augmented reality games that keep the player and wider community safe during play.
Michael Barlow

Trust

Frontmatter

Open Access

Chapter 8. The Role of Trust in Human-Robot Interaction

Abstract
As robots become increasingly common in a wide variety of domains—from military and scientific applications to entertainment and home use—there is an increasing need to define and assess the trust humans have when interacting with robots. In human interaction with robots and automation, previous work has discovered that humans often have a tendency to either overuse automation, especially in cases of high workload, or underuse automation, both of which can make negative outcomes more likely. Frthermore, this is not limited to naive users, but experienced ones as well. Robotics brings a new dimension to previous work in trust in automation, as they are envisioned by many to work as teammates with their operators in increasingly complex tasks. In this chapter, our goal is to highlight previous work in trust in automation and human-robot interaction and draw conclusions and recommendations based on the existing literature. We believe that, while significant progress has been made in recent years, especially in quantifying and modeling trust, there are still several places where more investigation is needed.
Michael Lewis, Katia Sycara, Phillip Walker

Open Access

Chapter 9. Trustworthiness of Autonomous Systems

Abstract
Effective robots and autonomous systems must be trustworthy. This chapter examines models of trustworthiness from a philosophical and empirical perspective to inform the design and adoption of autonomous systems. Trustworthiness is a property of trusted agents or organisations that engenders trust in other agent or organisations. Trust is a complex phenomena defined differently depending on the discipline. This chapter aims to bring different approaches under a single framework for investigation with three sorts of questions: Who or what is trustworthy?–metaphysics. How do we know who or what is trustworthy?–epistemology. What factors influence what or who should we trust?–normativity. A two-component model of trust is used that incorporates competence (skills, reliability and experience) and integrity (motives, honesty and character). It is supposed that human levels of competence yield the highest trust whereas trust is reduced at sub-human and super-human levels. The threshold for trustworthiness of an agent or organisation in a particular context is a function of their relationship with the truster and potential impacts of decisions. Building trustworthy autonomous systems requires obeying the norms of logic, rationality and ethics under pragmatic constraints–even though there is disagreement on these principles by experts. Autonomous systems may need sophisticated social identities including empathy and reputational concerns to build human-like trust relationships. Ultimately transdisciplinary research drawing on metaphysical, epistemological and normative human and machine theories of trust are needed to design trustworthy autonomous systems for adoption.
S. Kate Devitt

Open Access

Chapter 10. Trusted Autonomy Under Uncertainty

Abstract
The relationship between trust and uncertainty has not been fully developed in current frameworks on trust, including trust in autonomous systems. This chapter presents an investigation of this relationship. It begins with a survey of trust and distrust in general, followed by a focus on human-robot interaction (HRI). Thereafter, the roles of uncertainty in trust and distrust are elucidated, and the impacts of different kinds and sources of uncertainty are elaborated.
Michael Smithson

Open Access

Chapter 11. The Need for Trusted Autonomy in Military Cyber Security

Abstract
Information systems in the early 21st Century have become a critical enabler of increased value to the business, or as people in Defence might call a ‘force multiplier’.
Andrew Dowse

Open Access

Chapter 12. Reinforcing Trust in Autonomous Systems: A Quantum Cognitive Approach

Abstract
We investigated if an autonomous system can be provided with reasoning that maintains trust between human and system even when human and autonomous system reach discrepant conclusions. Tversky and Kahneman’s research [27] and the vast literature following it distinguishes two modes of human decision making: System 1, which is fast, emotional, and automatic, and System 2 which is slower, more deliberative, and more rational. Autonomous systems are thus far endowed with System 2. So when interacting with such a system, humans may follow System 1 unawares that their autonomous partner follows System 2. This can easily confuse the user when a discrepant decision is reached, eroding their trust in the autonomous system. Hence we investigated if trust in the message could interfere with trust its source, namely the autonomous system. For this we presented participants with images that might or might not be genuine, and found that they often distrusted the image (e.g., as photoshopped) when they distrusted its content. We present a quantum cognitive model that explains this interference. We speculate that enriching an autonomous system with this model will allow it to predict when its decisions may confuse the user, take pro-active steps to prevent this, and with it reinforce and maintain trust in the system.
Peter D. Bruza, Eduard C. Hoenkamp

Open Access

Chapter 13. Learning to Shape Errors with a Confusion Objective

Abstract
Errors are the enemy of classification systems, so minimising the total probability of error is an understandable objective in statistical machine learning classifiers. However, for open-world application in trusted autonomous systems , not all errors are equal in terms of their consequences. So, the ability for users and designers to define an objective function that distributes errors according to preference criteria might elevate trust. Previous approaches in cost-sensitive classification have focussed on dealing with distribution imbalances by cost weighting the probability of classification. A novel alternative is proposed that learns a ‘confusion objective’ and is suitable for integration with modular Deep Network architectures. The approach demonstrates an ability to control the error distribution in training of supervised networks via back-propagation for the penalty of an increase in total errors. Theory is developed for the new confusion objective function and compared with cross-entropy and squared loss objectives. The capacity for error shaping is demonstrated via a range of empirical experiments using a shallow and deep network. The classification of handwritten digits from up to three independent databases demonstrates desired error performance is maintained across unforeseen data distributions. Some significant and unique forms of error control are demonstrated and their limitations investigated.
Jason Scholz

Open Access

Chapter 14. Developing Robot Assistants with Communicative Cues for Safe, Fluent HRI

Abstract
The Collaborative Advanced Robotics and Intelligent Systems (CARIS) laboratory at the University of British Columbia studies the development of robotic systems that are capable of autonomous human-robot interaction. This chapter will provide an overview of our laboratory’s activities and methodologies. We first discuss a recently-concluded multi-institutional three year project to develop autonomous robot assistants which aid in assembly operations at manufacturing facilities. Next we discuss the primary methodology employed by our laboratory, by which we identify communicative cues used in interactions between people, describe these cues in detail, then implement and test them on robots. This is followed by an overview of recent communicative cue studies carried out by our group and our collaborators. We conclude by discussing current and future work.
Justin W. Hart, Sara Sheikholeslami, Brian Gleeson, Elizabeth Croft, Karon MacLean, Frank P. Ferrie, Clément Gosselin, Denis Laurandeau

Trusted Autonomy

Frontmatter

Open Access

Chapter 15. Intrinsic Motivation for Truly Autonomous Agents

Abstract
In this chapter, I argue that agents need human-like intrinsic motivation in order to achieve true autonomy, which is especially important for dealing with complex, uncertain, or unpredictable environments. A computational cognitive architecture is presented that incorporates not only cognitive capabilities necessary for the functioning of agents but also intrinsic (as well as derived) motivation for agents. With this model, an agent is able to function properly and autonomously in complex environments, as demonstrated by a wide range of simulations.
Ron Sun

Open Access

Chapter 16. Computational Motivation, Autonomy and Trustworthiness: Can We Have It All?

Abstract
Computational motivation—such as curiosity, novelty-seeking, achievement, affiliation and power motivation-facilitates open-ended goal generation by artificial agents and robots. This further supports diversity, adaptation and cumulative, life-long learning by machines. However, as machines acquire greater autonomy, this may begin to affect human perception of their trustworthiness. Can machines be self-motivated, autonomous and trustworthy? This chapter examines the impact of self-motivated autonomy on trustworthiness in the context of intrinsically motivated agent swarms.
Kathryn Merrick, Adam Klyne, Medria Hardhienata

Open Access

Chapter 17. Are Autonomous-and-Creative Machines Intrinsically Untrustworthy?

Abstract
Given what has been discovered in the case of human cognition, this principle seems plausible: An artificial agent that is both autonomous (A) and creative (C) will tend to be, from the viewpoint of a rational, fully informed agent, (U) untrustworthy . After briefly explaining the intuitive, internal structure of this disturbing (in the context of the human sphere) principle, we provide a more formal rendition of the principle designed to apply to the realm of intelligent artificial agents. The more-formal version makes use of some basic structures available in one of our cognitive-event calculi, and can be expressed as a (confessedly — for reasons explained — naïve) theorem. We prove the theorem, and provide simple demonstrations of it in action, using a novel theorem prover (ShadowProver). We end by pointing toward some future defensive engineering measures that should be taken in light of the theorem.
Selmer Bringsjord, Naveen Sundar Govindarajulu

Open Access

Chapter 18. Trusted Autonomous Command and Control

Abstract
This chapter will use a demonstration scenario, with intersecting vignettes, to investigate a path to where, by 2030, the promise of autonomous systems will have been realised. Like many steps humans have taken in technological evolution, some will be deliberate and cautious whereas others will happen out of necessity - perceived or real. Looking back through historical scenarios can provide an in-sight to the steps taken, their triggering events and the temporal factors, in order to identify potential futures. The crucible of conflict has always been a fertile ground for evolutionary change, particular those pushing the boundaries of moral and ethical thinking. From the flimsy flying machines at the beginning of World War One humanity saw the development of transcontinental aircraft and rockets in a little over three decades, culminating in the first operational use of an atomic weapon. High levels of digitisation and automation became central to the most capable weapon systems where human interaction could often result in degraded system performance, occasionally with disastrous consequences. Ongoing war-fare and disregard for the global rules based order enabled an environment where the combining of cyber and social media capabilities developed into crude but effective ‘trusted autonomous command and control’ system. Like all such steps taken before, this was embraced by global major powers and exploited to its maximum potential.
Noel Derwort

Open Access

Chapter 19. Trusted Autonomy in Training: A Future Scenario

Abstract
Being able to trust your teacher has been a pivotal assumption within the training systems. The advent of autonomous systems capable of delivering training in innovative and traditional ways creates a number of questions. The premise of this book allows us to examine how autonomous systems, that is non-human, will impact training and learning environments. The following section seeks to explore the future of trusted autonomy within a training context through both an extrapolation of current trends and creative thought.
Leon D. Young

Open Access

Chapter 20. Future Trusted Autonomous Space Scenarios

Abstract
This chapter describes the nature of the space environment that makes autonomous space systems a desirable application; describes the various types of space activities in near-Earth and deep space missions , and examples of autonomous systems deployed in space to date; outlines the current state-of-the-art of the intersection between trusted autonomous systems and autonomous space systems; and then presents a variety of possible future trusted autonomous space scenarios.
Russell Boyce, Douglas Griffin

Open Access

Chapter 21. An Autonomy Interrogative

Abstract
This chapter considers autonomy as an economic concern, meaning as a matter of determining the outcomes from different ways of allocating scarce resources in a social setting. Specifically, it concerns the outcomes from decision-making mechanisms subject to capital rationing - constraints on available resources - under conditions of fundamental uncertainty. This kind of uncertainty has long been a topic of economic study: epistemic uncertainty distinguishes stochastic risk from uncertainty, while ontological uncertainty equates to non-stationarity and non-regularity in a dynamical systems sense. I also argue that this deeper ontological uncertainty is a manifestation of mathematical incompleteness and unsolvability, which are inherent limitations in reasoning due to logical paradox. Given that non-linear dynamics classifies different degrees of uncertainty in different classes, I propose plasticity as the maximum class that an organism, system, organisation or thing may successfully handle, in the sense of surviving in an environment characterised by that class for better than logarithmic time. Machines that manifest plasticity beyond the most benign classes of uncertainty may then be referred to as autonomous, distinguishing them from automations that rely on strong prediction. Economics also studies mechanisms for handling uncertainty and their system-wide as well as individual outcomes. Paradoxically, uncertainty can easily result from the very measures that were intended to deal with uncertainty, so although a strategy might make completely rational sense from the point of view of a single agent, it can just as easily produce dangerously unstable and unpredictable system-wide outcomes. I focus on a certain class of financial strategies in particular, known as ‘barbell’ and ‘dumbbell’ strategies - which divide investment into hedges against intolerable failure and opportunity bets where failure is tolerable - for their apparent applicability across different classes of uncertainty. At the centre of this picture is a requirement for a theory of self by which an agent can determine failure sensitivity and affordable opportunity in high uncertainty environments under partially observable hard resource limits. The limits of self-knowledge together with the complexity of decision-making under hard capital rationing with the possibility of unexpected budget changes appears to imply that autonomous problem solving must be intrinsically social.
Darryn J. Reid

Backmatter

Weitere Informationen