A few-shot learning-based dual-input neural network for complex spectrogram recognition system with millimeter-wave radar

Graph data-driven machine learning methods for human activity recognition (HAR) have achieved success recently using sufficient data. In the realm of everyday life, we encounter a notable challenge: the scarcity of labeled radar samples. This limitation is compounded by the stark disparities in data distribution between simulated and measured activity domains. In this article, a generalized graph contrastive learning framework (DSMFT-Net) incorporated with Boulic-Thalmann simulation model for few-shot HAR is proposed. DSMFT-Net combines a clustering strategy with contrastive learning to develop a robust, domain-invariant feature representation. Particularly, the method divided into two phases: single radar range Doppler spectrogram prototypical contrast, enhancing the classification discriminative features by improving the compaction of prototypes and instances within a domain. Then, cross prototypical contrast of simulated and measured radar range Doppler spectrogram domain, focuses on discovering domain-invariant features through prototype-instance matching and proximity exploration. Moreover, mutual information maximization ensures the reliability of predictions, while pseudo-label information aids in self-supervised contrastive pre-training by comparing positive and negative sample pairs. The effectiveness of the model is empirically validated through testing conducted in both open and complex office environments. The experimental results indicate that the proposed method achieves an average accuracy of 93.3% under 5-shot setting and 96.5% under 10-shot setting across six human activity recognition tasks. These findings highlight the effectiveness of the proposed method in achieving high performance even with limited labeled data.