Addressing the vulnerability of deep neural networks (DNNs) has attracted significant attention in recent years. While recent studies on adversarial attack and defense mainly reside in a single image, few efforts have been made to perform temporal attacks against video sequences. As the temporal consistency between frames is not considered, existing adversarial attack approaches designed for static images do not perform well for deep object tracking. In this work, we generate adversarial examples on top of video sequences to improve the tracking robustness against adversarial attacks under white-box and black-box settings. To this end, we consider motion signals when generating lightweight perturbations over the estimated tracking results frame-by-frame. For the white-box attack, we generate temporal perturbations via known trackers to degrade significantly the tracking performance. We transfer the generated perturbations into unknown targeted trackers for the black-box attack to achieve transferring attacks. Furthermore, we train universal adversarial perturbations and directly add them into all frames of videos, improving the attack effectiveness with minor computational costs. On the other hand, we sequentially learn to estimate and remove the perturbations from input sequences to restore the tracking performance. We apply the proposed adversarial attack and defense approaches to state-of-the-art tracking algorithms. Extensive evaluations on large-scale benchmark datasets, including OTB, VOT, UAV123, and LaSOT, demonstrate that our attack method degrades the tracking performance significantly with favorable transferability to other backbones and trackers. Notably, the proposed defense method restores the original tracking performance to some extent and achieves additional performance gains when not under adversarial attacks.

近年来，解决深度神经网络（DNN）的脆弱性引起了广泛关注。虽然最近关于对抗性攻击和防御的研究主要集中在单个图像上，但很少有人致力于针对视频序列进行时间攻击。由于没有考虑帧之间的时间一致性，现有的针对静态图像设计的对抗性攻击方法对于深度对象跟踪效果不佳。在这项工作中，我们在视频序列之上生成对抗性示例，以提高白盒和黑盒设置下针对对抗性攻击的跟踪鲁棒性。为此，我们在对估计的跟踪结果逐帧生成轻量级扰动时考虑运动信号。对于白盒攻击，我们通过已知的跟踪器生成时间扰动，以显着降低跟踪性能。我们将产生的扰动转移到未知的目标跟踪器中进行黑盒攻击，以实现转移攻击。此外，我们训练通用的对抗性扰动并将其直接添加到视频的所有帧中，以较小的计算成本提高攻击效率。另一方面，我们依次学习估计和消除输入序列中的扰动以恢复跟踪性能。我们将所提出的对抗性攻击和防御方法应用于最先进的跟踪算法。对大型基准数据集（包括 OTB、VOT、UAV123 和 LaSOT）的广泛评估表明，我们的攻击方法显着降低了跟踪性能，并且具有向其他骨干网和跟踪器的良好可转移性。 值得注意的是，所提出的防御方法在一定程度上恢复了原始的跟踪性能，并在不受对抗性攻击时实现了额外的性能增益。