Self-supervised saliency-guided dual-attention fusion for benign-malignant thyroid nodule diagnosis in ultrasound

Accurate benign–malignant diagnosis of thyroid nodules from ultrasound images remains challenging due to low contrast, speckle noise, and substantial inter-center variability. Most existing deep learning approaches operate directly on raw ultrasound images and rely on fully supervised learning, lacking explicit mechanisms to leverage lesion-level localization for diagnostic guidance. Although some methods incorporate segmentation to assist classification, they typically depend on fully supervised pixel-level annotations, which are often unavailable in real-world clinical settings and limit generalization to data from different clinical centers. To address these limitations, we propose a self-supervised saliency-guided dual-attention fusion framework for thyroid nodule diagnosis in ultrasound images. The proposed method learns a saliency mask to highlight diagnostically informative regions and employs self-supervised reconstruction, boundary consistency, and affinity constraints to regularize saliency learning without requiring pixel-level annotations. In addition, a lightweight dual-attention fusion module is designed to integrate original and saliency-enhanced features. Unlike conventional attention mechanisms, this module combines variance-guided channel attention with patch-wise spatial attention, enabling selective emphasis on informative channels in the original image while capturing lesion-specific contextual features guided by the saliency map. This design facilitates the learning of more discriminative representations for benign–malignant classification. Extensive experiments conducted on three multi-center thyroid ultrasound datasets, under both single-center and cross-center evaluation protocols, demonstrate that the proposed framework consistently outperforms state-of-the-art methods, with particularly notable gains in multi-center generalization performance. Code is available at https://github.com/guangguangLi/MultiModalClassifier.