Hi-Net: Hybrid-Fusion Network for Multi-Modal MR Image Synthesis

Tao Zhou, Huazhu Fu, Geng Chen, Jianbing Shen, Ling Shao

科研成果: 期刊稿件文章同行评审

289 引用 (Scopus)

摘要

Magnetic resonance imaging (MRI) is a widely used neuroimaging technique that can provide images of different contrasts (i.e., modalities). Fusing this multi-modal data has proven particularly effective for boosting model performance in many tasks. However, due to poor data quality and frequent patient dropout, collecting all modalities for every patient remains a challenge. Medical image synthesis has been proposed as an effective solution, where any missing modalities are synthesized from the existing ones. In this paper, we propose a novel Hybrid-fusion Network (Hi-Net) for multi-modal MR image synthesis, which learns a mapping from multi-modal source images (i.e., existing modalities) to target images (i.e., missing modalities). In our Hi-Net, a modality-specific network is utilized to learn representations for each individual modality, and a fusion network is employed to learn the common latent representation of multi-modal data. Then, a multi-modal synthesis network is designed to densely combine the latent representation with hierarchical features from each modality, acting as a generator to synthesize the target images. Moreover, a layer-wise multi-modal fusion strategy effectively exploits the correlations among multiple modalities, where a Mixed Fusion Block (MFB) is proposed to adaptively weight different fusion strategies. Extensive experiments demonstrate the proposed model outperforms other state-of-the-art medical image synthesis methods.

源语言英语
文章编号9004544
页(从-至)2772-2781
页数10
期刊IEEE Transactions on Medical Imaging
39
9
DOI
出版状态已出版 - 9月 2020
已对外发布

指纹

探究 'Hi-Net: Hybrid-Fusion Network for Multi-Modal MR Image Synthesis' 的科研主题。它们共同构成独一无二的指纹。

引用此