Propagation rectified attack: on improving adversarial transferability

In the context of enhancing adversarial transferability along the line of surrogate refinement, this paper investigates transferable black-box attacks and proposes propagation rectified attack (PRA), which rectifies both the forward and backward propagation of the surrogate. Specifically, on rectifying the forward propagation, we develop multi-scale feature rectification (MSFR), which applies the feature rectifications to different levels of features, encouraging the forward propagation to be in the proper status of adversarial optimization, and highlighting the necessity and benefits of multi-scale decays for enhancing transferability, which has been ignored by existing studies. Additionally, for the backward propagation, existing studies only pursue the smoothness of the alternative activation derivative. Instead, we derive a more feasible and comprehensive conclusion. First, the derivative of the activation should be non-negative and monotonic, maintaining the gradient integrity. Besides, its second derivative should have a certain degree of magnitude near zero. Based on these findings, we further propose adaptive activation rectification (AAR), which takes the specificity of the features from each layer into account, thereby building a more effective activation alternative. Our evaluations are performed on two widely adopted datasets: ImageNet (with average gains of +13.85% over ten classical CNN models and +15.38% over six non-conventional-CNN models) and CIFAR-10 (with average gains of +5.5%). Codes will be released at https://github.com/phyyyy/PRA.