RIFormer: Learning Rotation-Invariant Features Via Transformer

Recently, Transformers have been widely used in many computer vision tasks and have shown promising results. However, like convolutional neural networks (CNNs), Transformers cannot handle rotational variations well, thus hindering its further application in the field of remote sensing. In this paper, we design a rotation-invariant Transformer (RIFormer) to alleviate the abovementioned problem. Moreover, we propose a novel rotation-invariant position embedding (RIPE) to encode positional information of features, and this position-dependent features learned by RIPE is robust to rotations. The experimental results show that proposed RIFormer with RIPE can effectively learn rotation-invariant features compared to the state-of-the-art methods with limited parameters. We provide an open-source implementation of our method. It is publicly available at https://github.com/psychAo/RIFormer.