SFRNet: Fine-Grained Oriented Object Recognition via Separate Feature Refinement

Gong Cheng, Qingyang Li, Guangxing Wang, Xingxing Xie, Lingtong Min, Junwei Han

Research output: Contribution to journalArticlepeer-review

73 Scopus citations

Abstract

Fine-grained oriented object recognition (FGO2R) is a practical need for intellectually interpreting remote sensing images. It aims at realizing fine-grained classification and precise localization with oriented bounding boxes, simultaneously. Our considerations for the task are general but decisive: 1) the extraction of subtle differences carries a big weight in differentiating fine-grained classes and 2) oriented localization prefers rotation-sensitive features. In this article, we propose a network with separate feature refinement (SFRNet), in which two transformer-based branches are designed to perform function-specific feature refinement for fine-grained classification and oriented localization, separately. To highlight the discriminative information advantageous to fine-grained classification, we propose a spatial and channel transformer (SC-Former) to capture both the long-range spatial interactions and the key correlations hidden in the feature channels. Besides, we design a multiregion of interest (RoI) loss (MRL) following the protocol of deep metric learning to enhance the separability of fine-grained classes further. For oriented localization, we integrate the oriented response convolution with the transformer structure (namely, OR-Former) to assist in encoding rotation information during regression. Extensive experimental results validate the effectiveness and robustness of our SFRNet. Without bells and whistles, our SFRNet achieves the state-of-the-art performance on the large-scale FAIR1M datasets (FAIR1M-1.0 and FAIR1M-2.0). Code will be available at https://github.com/Ranchosky/SFRNet.

Original languageEnglish
Article number5610510
JournalIEEE Transactions on Geoscience and Remote Sensing
Volume61
DOIs
StatePublished - 2023

Keywords

  • Deep metric learning
  • fine-grained classification
  • oriented object detection
  • vision transformer

Fingerprint

Dive into the research topics of 'SFRNet: Fine-Grained Oriented Object Recognition via Separate Feature Refinement'. Together they form a unique fingerprint.

Cite this