A lightweight spatial-temporal network with frequency-range fusion for edge-enabled FMCW radar human activity recognition

In internet of things (IoT) applications, contactless human activity recognition (HAR) is crucial in scenarios such as smart healthcare, elderly monitoring, and intelligent security. However, existing deep learning methods for FMCW radar-based HAR generally have high computational complexity and memory usage, making them difficult to deploy on resource-constrained IoT edge devices, and they lack robustness in handling cross-scenario and noisy environments. This paper proposes lightweight frequency-range map based spatio-temporal network (LFRST) to address these limitations. Specifically, our approach involves: (1) Time-frequency analysis is applied to convert non-stationary radar echoes into structured range-Doppler spectrograms, effectively capturing motion signatures across frequency, range, and temporal domains; (2) designing a 16 × 16 large-stride patch embedding layer to not only reduce token count but also expand receptive fields for enhanced feature representation; and (3) introducing an efficient single-head self-attention (SHSA) mechanism, rather than multi-head self-attention in standard Transformers, to reduce structural complexity and memory access overhead. LFRST achieves 99.3% accuracy on six human activities in a complex office scenario and is further validated on the mmDoppler and DIAT-μRadHAR datasets through cross-domain experiments. Additionally, extensive evaluations demonstrate that LFRST outperforms state-of-the-art methods in terms of accuracy-speed trade-off. Importantly, LFRST successfully deploys on Raspberry Pi 4B achieving 46 FPS at 2.6 W, as well as on Jetson Nano and STM32H7 MCU, showcasing its versatility across different IoT hardware environments.