Object detection is a fundamental task in various applications ranging from autonomous driving to intelligent security systems. However, recognition of a person can be hindered when their clothing is decorated with carefully designed graffiti patterns, leading to the failure of object detection. To achieve greater attack potential against unknown black-box models, adversarial patches capable of affecting the outputs of multiple-object detection models are required. While ensemble models have proven effective, current research in the field of object detection typically focuses on the simple fusion of the outputs of all models, with limited attention being given to developing general adversarial patches that can function effectively in the physical world. In this paper, we introduce the concept of energy and treat the adversarial patches generation process as an optimization of the adversarial patches to minimize the total energy of the “person” category. Additionally, by adopting adversarial training, we construct a dynamically optimized ensemble model. During training, the weight parameters of the attacked target models are adjusted to find the balance point at which the generated adversarial patches can effectively attack all target models. We carried out six sets of comparative experiments and tested our algorithm on five mainstream object detection models. The adversarial patches generated by our algorithm can reduce the recognition accuracy of YOLOv2 and YOLOv3 to 13.19% and 29.20%, respectively. In addition, we conducted experiments to test the effectiveness of T-shirts covered with our adversarial patches in the physical world and could achieve that people are not recognized by the object detection model. Finally, leveraging the Grad-CAM tool, we explored the attack mechanism of adversarial patches from an energetic perspective.

中文翻译：

---

DOEPatch：用于生成对抗性补丁的动态优化集成模型


对象检测是各种应用的基本任务，从自动驾驶到智能安全系统。但是，当一个人的衣服上装饰着精心设计的涂鸦图案时，可能会阻碍对人的识别，从而导致物体检测失败。为了对未知的黑盒模型实现更大的攻击潜力，需要能够影响多目标检测模型输出的对抗性补丁。虽然集成模型已被证明是有效的，但目前对象检测领域的研究通常集中在所有模型输出的简单融合上，而对开发可以在物理世界中有效运作的通用对抗补丁的关注有限。在本文中，我们介绍了能量的概念，并将对抗性补丁的生成过程视为对抗性补丁的优化，以最小化 “人” 类别的总能量。此外，通过采用对抗性训练，我们构建了一个动态优化的集成模型。在训练过程中，调整被攻击目标模型的权重参数，以找到生成的对抗补丁可以有效攻击所有目标模型的平衡点。我们进行了六组比较实验，并在 5 个主流目标检测模型上测试了我们的算法。我们的算法生成的对抗补丁可以将 YOLOv2 和 YOLOv3 的识别准确率分别降低到 13.19% 和 29.20%。此外，我们还进行了实验，以测试在物理世界中覆盖有对抗性补丁的 T 恤的有效性，并且可以实现对象检测模型无法识别人。 最后，利用 Grad-CAM 工具，我们从能量角度探讨了对抗性补丁的攻击机制。