MM-DCNet: A Multi-Scale and Multi-MOdality Dynamic Convolutional Network for Lung Cancer Subtypes Classification

Gege Ma, Yuan Jin, Tianling Lyu, Geng Chen, Zhuoxuan Wu, Jiaqi Zhao, Wentao Zhu

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Early and accurate subtypes classifications for lung cancer patients is critical for the following-up treatments and prognosis. CT scan, being the non-invasive and fast-imaging modality, is one of the most common used techniques for cancer diagnosis, thus, the CT-based automatic analysis systems are in high demand. However, the accuracy of such models are limited due to the relative low resolution of CT images. Clinically, pathological examination is regarded as "gold standard"in cancer diagnosis, so the introduction of such cellular-level information into CT-based model is expected to improve model's accuracy. However, the invasive pathological examination may not be applicable to all clinical scenarios, leading to the presence of numerous unbalanced multi-modality images, i.e., paired CT/pathology images as well as standalone CT images. In this work, we propose a novel classification model, i.e., multi-scale and multi-modality dynamic convolutional network (MM-DCNet), to assist the diagnosis of lung cancer subtypes using unbalanced CT and pathological images. Within the model, we designed a dynamic convolutional module that empowers the model to adaptively adjust its parameters according to different inputs, e.g., the paired multi-modality images and the single-modality image, and consequently exploit the value of all clinical datasets. Furthermore, we designed a contrastive learning module to acquire the cross-modality correlations from paired CT/pathological images and subsequently leverage such correlations as priors to lead the model to more accurate predictions even in the absence of pathology. Experiment results have demonstrated the superiority of our proposed model in lung cancer subtypes diagnosis.

Original languageEnglish
Title of host publicationIEEE International Symposium on Biomedical Imaging, ISBI 2024 - Conference Proceedings
PublisherIEEE Computer Society
ISBN (Electronic)9798350313338
DOIs
StatePublished - 2024
Event21st IEEE International Symposium on Biomedical Imaging, ISBI 2024 - Athens, Greece
Duration: 27 May 202430 May 2024

Publication series

NameProceedings - International Symposium on Biomedical Imaging
ISSN (Print)1945-7928
ISSN (Electronic)1945-8452

Conference

Conference21st IEEE International Symposium on Biomedical Imaging, ISBI 2024
Country/TerritoryGreece
CityAthens
Period27/05/2430/05/24

Keywords

  • CT
  • Image classification
  • multi-modality
  • neural network
  • unbalanced dataset

Fingerprint

Dive into the research topics of 'MM-DCNet: A Multi-Scale and Multi-MOdality Dynamic Convolutional Network for Lung Cancer Subtypes Classification'. Together they form a unique fingerprint.

Cite this