Attentional Feature Erase: Towards task-wise transferable adversarial attack on cloud vision APIs

Recent works have shown that adversarial examples (AEs) can attack and successfully transfer across various neural networks, highlighting the potential danger they pose. However, current approaches that focus on task-specific loss functions may not be as effective across different tasks. Additionally, the use of cloud APIs in practice, which often involve combining multiple tasks, also weakens the effectiveness of existing attacks. To address these issues, we propose a method called Attentional Feature Erase, which is a task-agnostic attack with improved cross-task transferability and effectiveness on computer vision-based cloud APIs. We view the transferability of AEs as a latent contribution for each layer of deep neural networks. By focusing on the intermediate layers of model backbones and reducing high-value features in each intermediate feature map, we are able to maximize the attack performance. Additionally, to better aggregate the gradients and generate adversarial perturbations during backward propagation, Transferability Regularizer is proposed to calculate the attention heatmap for each intermediate feature map and systematically combine the gradients. Comprehensive set of experiments on the Google Cloud Vision APIs and public available datasets (i.e. ImageNet, PASCAL VOC and MS COCO) show that the proposed AFE attack is more effective and has better transferability compared to the state-of-the-art baselines.