Cognitive Security

A System-Scientific Approach

verfasst von: Linan Huang, Quanyan Zhu

Verlag: Springer International Publishing

Buchreihe : SpringerBriefs in Computer Science

Enthalten in: Springer Professional "Wirtschaft+Technik" , Springer Professional "Technik" , Springer Professional "Wirtschaft"

Einloggen, um Zugang zu erhalten

Über dieses Buch

This book presents the latest research in cognitive security, a rapidly emerging field that addresses the vulnerabilities in human behavior and cognition that can lead to Cyber-Physical Systems (CPS) compromise. This book demonstrates that as adversaries increasingly use manipulative and deceptive information to disrupt human cognitive processes, including sensation, attention, memory, and mental operations, humans are misled into fallacious reasoning and manipulated decisions that can lead to system-level meltdown. Cognitive security aims to protect humans from the exploitation of cognitive vulnerabilities, help them make informed decisions that are free from manipulation and undue influence, and mitigate the aggravating risk in the ensuing steps of the attacker’s kill chain.

This book offers solutions that work across different fields, such as psychology, neuroscience, data science, social science, and game theory, to deal with cognitive threats. It guides the reader through the core ideas with figures, real-life examples, and case studies. Moreover, it formally defines all research questions, presents the results using mathematical theorems and proofs, and obtains insights through numerical validation.

This book provides a self-contained and brief overview of essential system-scientific tools for modeling, analyzing, and mitigating cognitive vulnerabilities. The concepts of human cognitive capacities and cognitive vulnerabilities are formally discussed, followed by two case studies in the scenarios of reactive and proactive attention vulnerabilities. This book provides insights and applications on this transdisciplinary topic, with the goal of motivating future research in this emerging area and pushing the frontier of human-technology convergence. This book is a valuable reference for researchers and advanced-level students studying or working in cognitive security and related fields. It is also useful for decision-makers, managers, and professionals working within these related fields.

Inhaltsverzeichnis

Frontmatter

Chapter 1. Introduction

Abstract

Human cognitive capacities and the needs of human-centric solutions for “Industry 5.0” make humans an indispensable component in Cyber-Physical Systems (CPSs), referred to as Human-Cyber-Physical Systems (HCPSs), where AI-powered technologies are incorporated to assist and augment humans. The close integration between humans and technologies in Sect. 1.1 and cognitive attacks in Sect. 1.2.4 poses emerging security challenges, where attacks can exploit vulnerabilities of human cognitive processes, affect their behaviors, and ultimately damage the HCPS.

Defending HCPSs against cognitive attacks requires a new security paradigm, which we refer to as “cognitive security” in Sect. 1.2.5. The vulnerabilities of human cognitive systems and the associated methods of exploitation distinguish cognitive security from “cognitive reliability” and give rise to a distinctive CIA triad, as shown in Sects. 1.2.5.1 and 1.2.5.2, respectively. Section 1.2.5.3 introduces cognitive and technical defense methods that deter the kill chain of cognitive attacks and harden the cognitive security. System scientific perspectives in Sect. 1.3 offer a promising direction to address the new challenges of cognitive security by developing quantitative, modular, multi-scale, and transferable solutions. Figure 1.1 illustrates the structure of Chap. 1.

Linan Huang, Quanyan Zhu

Chapter 2. System-Scientific Methods

Abstract

In Chap. 2, we briefly introduce essential system-scientific tools for modeling, analyzing, and mitigating cognitive vulnerabilities and cognitive attacks. Decision theory in Sect. 2.1 provides a scientific foundation of making decisions for single agents with different rationality levels under stochastic environments. Game theory is introduced in Sect. 2.2 to model the strategic interactions among multiple agents under several basic game settings and their associated Nash Equilibrium (NE) solution concepts. To address the challenges of incomplete information in decision-making and game modeling, we present two learning schemes in Sect. 2.3. These tools provide a system-scientific perspective to evaluate and reduce uncertainty in HCPSs, as illustrated by the blue and red lines in Fig. 2.1, respectively. We refer the readers to the notes at the end of each section for recent advances and relevant references.

Linan Huang, Quanyan Zhu

Chapter 3. Cognitive Capacities for Designing, Operating, Supervising, and Securing Complex Systems

Abstract

Despite the integration of a variety of “Industry 4.0” technologies, the active presence of human actors is required to bring their cognitive capabilities to Designing, Operating, Supervising, and Securing (DOSS) complex systems. Figure 3.1 illustrates the structure of Chap. 3. In particular, we discuss situation awareness in Sect. 3.1, problem solving in Sect. 3.2, and collaboration in Sect. 3.3 as the three essential cognitive capabilities that distinguish a human actor from a traditional cyber-physical component in the above DOSS procedures. Examples of human roles in four DOSS procedures are introduced to illustrate these three classes of cognitive capabilities and are mapped onto the four quadrants in Fig. 3.2 based on their time-sensitivity and interactivity.

Linan Huang, Quanyan Zhu

Chapter 4. Review of System-Scientific Perspectives for Analysis, Exploitation, and Mitigation of Cognitive Vulnerabilities

Abstract

Chapter 3 elaborates on three critical types of human cognitive capabilities to fulfill four classes of CPS tasks in Fig. 3.2. For all of its advantages, the active presence of human cognition also brings vulnerabilities. Compared to computer programs and robots that strictly follow the algorithms and retain the same level of performance, human operators may violate security procedures or be prone to errors due to misaligned incentives, herding effects, inattention, fatigue, and bounded rationality.

In Chap. 4, we classify cognitive vulnerabilities into innate vulnerabilities in Sect. 4.1 and acquired vulnerabilities in Sect. 4.2 based on whether they can be mitigated through short-term external interference, including security training and mechanism design in Fig. 1.8. For each cognitive vulnerability, we first illustrate its impact on HCPSs and how cognitive attacks can exploit it. Then, we present system-scientific perspectives to characterize the vulnerability, the attacks, and the defense methods in different security scenarios, which focus on the computational aspects of vulnerability analysis, exploitation, and mitigation in the literature.

Linan Huang, Quanyan Zhu

Chapter 5. ADVERT: Defending against Reactive Attention Attacks

Abstract

Following the definition in Sect. 1.2.3.2, phishing can be a typical class of reactive attention attacks that exploit inattention to evade detection. This chapter proposes ADVERT, a human-technical solution that generates adaptive visual aids in real-time to prevent users from inadvertence and reduce their susceptibility to phishing attacks. Based on eye-tracking data, we extract visual states and attention states as system-level sufficient statistics to characterize the user’s visual behaviors and attention status. By adopting a data-driven approach and two learning feedback of different time scales, this work lays out a theoretical foundation to analyze, evaluate, and particularly modify humans’ attention processes while they vet and recognize phishing emails. We corroborate the effectiveness, efficiency, and robustness of ADVERT through a case study based on the data set collected from human subject experiments conducted at New York University. The results show that the visual aids can statistically increase the attention level and improve the accuracy of phishing recognition from 74.6% to a minimum of 86%. The meta-adaptation can further improve the accuracy to 91.5% (resp. 93.7%) in less than 3 (resp. 50) tuning stages.

Linan Huang, Quanyan Zhu

Chapter 6. RADAMS: Defending Against Proactive Attention Attacks

Abstract

Following the definition in Sect. 1.2.3.2, in this chapter, we identify and formally define a new type of proactive attention attacks called Informational Denial-of-Service (IDoS) attacks that generate a large volume of feint attacks to overload human operators and hide real attacks among feints. We incorporate human factors (e.g., levels of expertise, stress, and efficiency) and empirical psychological results (e.g., the Yerkes–Dodson law and the sunk cost fallacy) to model the operators’ attention dynamics and their decision-making processes along with the real-time alert monitoring and inspection. To assist human operators in dismissing the feints and escalating the real attacks timely and accurately, we develop a Resilient and Adaptive Data-driven alert and Attention Management Strategy (RADAMS) that de-emphasizes alerts selectively based on the abstracted category labels of the alerts. RADAMS uses Reinforcement Learning (RL) to achieve a customized and transferable design for various human operators and evolving IDoS attacks. The integrated modeling and theoretical analysis lead to the Product Principle of Attention (PPoA), fundamental limits, and the tradeoff among crucial human and economic factors.

Linan Huang, Quanyan Zhu

Chapter 7. Summary and Conclusions

Abstract

This book has introduced emerging cyber threats that exploit human vulnerabilities to obtain initial credentials from human users. Cognitive security is a primary concern that needs to be addressed in HCPSs. We have presented a system-scientific foundation that builds on and bridges the fields of psychology, neuroscience, data science, decision theory, and game theory. Such a foundation has led to transdisciplinary socio-technical solutions that protect humans from cognitive security threats and improve the resiliency of HCPSs. In this chapter, we summarize the book, discuss several insights, explore potential applications, and suggest potential directions for future work.

Linan Huang, Quanyan Zhu

Titel: Cognitive Security
verfasst von: Linan Huang
Quanyan Zhu
Verlag: Springer International Publishing
Electronic ISBN: 978-3-031-30709-6
Print ISBN: 978-3-031-30708-9
DOI: https://doi.org/10.1007/978-3-031-30709-6