Transfer Attention-Guided Multi-Receptive Field Network for Multi-Modality Cardiac Image Segmentation

Existing whole heart segmentation algorithms usually combine 3D Convolutional Neural Networks (3D CNNs) with Transformers, for the purpose of capturing local and global features. However, traditional CNNs with a fixed size of receptive field cannot capture long-range contextual information. Transformers have been widely used to establish dependencies on global information, despite this, they greatly increase the computational complexity. To mitigate these challenges, we propose a hybrid paradigm, called Transfer Attention-Guided MultiReceptive Field Network (TAMRNet), to boost the representation quality for multi-modality cardiac image segmentation. In TAMRNet, the novel adaptive-scale depthwise convolution module adeptly preserves the inherent inductive biases of convolution while concurrently amplifying the network's ability to establish dependencies on long-range contextual information. Besides, a novel attention mechanism called Transfer Attention is developed to establish dependencies on global information. Transfer Attention avoids the direct similarity calculation of Q and K by introducing the Transfer tokens, and thus dramatically decreases the computational cost. The proposed TAMRNet is tested on the MM-WHS 2017 challenge dataset, achieving the average Dice scores of 93.7% and 82.2% on the CT and MRI datasets respectively. Extensive experimental results prove that our proposed method achieves superior performances in comparison with state-of-the-art methods.