Constructing a Multi-Modal based Underwater Acoustic Target Recognition Method with a Pre-trained Language-Audio Model

Underwater Acoustic Target Recognition (UATR) aims to accurately identify radiated acoustic signals from ships in complex maritime environments. The challenges of this task lay in how to explore discriminative representation from complex and limited acoustic samples. Recently, various deep learning-based UATR methods have been proposed. However, their performance on real sonar-collected signals remains restricted. On one hand, most methods currently adopt different representation extraction strategies to extract features from acoustic signals such as time-frequency representation, wave representation, and joint representation. However, the limited feature representation capability and simple feature fusion strategies often limit the recognition performance improvement. On the other hand, they often overlook the knowledge gains brought by pre-trained models and the extraction of multi-feature semantic correlation knowledge. This leads to unsatisfactory performance and even overfitting issues. To mitigate these issues, this paper proposes a Multi-Feature Underwater Acoustic Target Recognition method (MF-UATR). It introduces a strongly generalized multi-modal pre-trained language-audio model and contrastive learning based feature-level fusion strategy to semantically guide and fuse multiple features. This strategy facilitates the model in learning prior knowledge and the semantic correlations between features thereby improving recognition performance. Additionally, we also considered the few-shot scenarios with extremely limited data, in which a Multi-Modal Few-Shot Underwater Acoustic Target Recognition (MMFS-UATR) scheme is proposed. It efficiently completes the few-shot underwater acoustic target recognition task by combining parameter-efficient fine-tuning techniques, semantic supervision strategy, and pre-trained MF-UATR. Extensive experiments on two public datasets, DeepShip and ShipsEar, demonstrate that the proposed frameworks achieve optimal target recognition performance under regular and few-shot settings.