4. Security in AI Systems

Artificial intelligence is no longer confined to labs as it is now embedded in finance, healthcare, and transportation, which means its security has become a serious issue. Recent frameworks for “trustworthy AI” emphasize that security is just as important as safety, fairness, or transparency. Still, research has repeatedly shown that even high-accuracy models can be deceived by tiny changes that humans hardly notice. A striking example is the altered stop sign that an autonomous car misread as a speed-limit sign, simply because of the presence of a few stickers. Other attack types target the training process like data poisoning can bias a model or quietly insert backdoors that remain dormant until a specific trigger is present (Liu et al. in Trojaning attack on neural networks. NDSS [10]). Model extraction, or “stealing,” allows adversaries to recreate proprietary models by querying APIs, as shown in cloud-based attacks. Privacy is also at stake like membership inference and model inversion can reveal whether a person’s data was part of training or even reconstruct sensitive attributes. To defend against the risks, researchers have explored adversarial training, feature squeezing, and backdoor detection like Neural Cleanse. Privacy-preserving approaches like differential privacy and federated learning with secure aggregation are also evolving, though they often reduce accuracy. Industry reports recommend robust lifecycle practices like data provenance, model signing, red teaming, and monitoring for us to mitigate supply chain and misuse risks. Toward the end of the chapter we look at arguments that AI security is not solved by one trick but it requires a layered strategy and cross-disciplinary governance, much like the trajectory of traditional cybersecurity.