PNAP-YOLO: An Improved Prompts-Based Naturalistic Adversarial Patch Model for Object Detectors

The rapid advancement of deep learning in computer vision has led to significant breakthroughs in various applications, from face recognition to autonomous driving. However, these models are vulnerable to adversarial examples, which can introduce perturbations leading to incorrect judgments and posing serious threats to security and reliability. This article addresses the limitations of traditional adversarial patches, which often have conspicuous patterns and are easily identifiable. It introduces a Prompt-based Natural Adversarial Patch (PNAP) generation method that leverages a pretrained Latent Diffusion Model (LDM) to create patches with specific semantic content. The method ensures the visual naturalness of the patches while maintaining high attack performance on detectors. The article provides a detailed overview of the PNAP generation process, including patch training, embedding, and loss function design. It also presents comprehensive evaluations of the PNAP's attack performance in both digital and physical world scenarios, as well as subjective evaluations of its naturalness. The results demonstrate that PNAP can significantly improve the naturalness of adversarial patches while delivering effective attacks, making it a valuable contribution to the field of adversarial machine learning.