Effcient two-dimensional direction finding algorithm for rectilinear sources under unknown mutual coupling

Digital communication signals in wireless systems may possess noncircularity, which can be used to enhance the degrees of freedom for direction-of-arrival (DOA) estimation in sensor array signal processing. On the other hand, the electromagnetic characteristics between sensors in uniform rectangular arrays (URAs), such as mutual coupling, may significantly deteriorate the estimation performance. To deal with this problem, a robust real-valued estimator for rectilinear sources was developed to alleviate unknown mutual coupling in URAs. An augmented covariance matrix was built up by extracting the real and imaginary parts of observations containing the circularity and noncircularity of signals. Then, the actual steering vector considering mutual coupling was reparameterized to make the rank reduction (RARE) property available. To reduce the computational complexity of two-dimensional (2D) spectral search, we individually estimated y-axis and x-axis direction-cosines in two stages following the principle of RARE. Finally, azimuth and elevation angle estimates were determined from the corresponding direction-cosines respectively. Compared with existing solutions, the proposed method is more computationally e_cient, involving real-valued operations and decoupled 2D spectral searches into twice those of one-dimensional searches. Simulation results verified that the proposed method provides satisfactory estimation performance that is robust to unknown mutual coupling and close to the counterparts based on 2D spectral searches, but at the cost of much fewer calculations.