Deep-unfolding transformer for DOA estimation of low frequency source under mismatched small aperture array

Direction-of-Arrival (DOA) estimation plays a key part in underwater observation and communication, but it is challenging for traditional model-based and deep-learning methods to achieve high resolution ability and accuracy with mismatched small-aperture array given low-frequency signals. To ensure fine performance under above-mentioned adverse conditions, this work combines Transformer with deep-unfolding learnable iterative shrinkage threshold algorithm (LISTA) and proposes a Deep-unfolding LISTA-Transformer (DIT) architecture. The proposed DIT first maps low-frequency covariance matrix into high-dimensional latent space, and then reconstructs the high-dimensional feature map and predicts surrogate high-frequency covariance matrix with Transformer module. The following LISTA layer utilizes the surrogate covariance matrix to predict high-resolution spatial spectrum based on sparse signal prior. Compared with existing deep-learning methods, DIT fully utilizes sparse prior, enabling much higher resolution and generalization ability. Extensive numerical simulations verify superior resolution and robustness of DIT, and experiment results further illustrate its practicability in complicated underwater environment with small aperture and low-frequency signal sources.