Enhancing long-range contextual interactions via frequency-aware directional shifts for medical image segmentation

Transformer-based models provide strong global context modeling for medical image segmentation, but their high computational cost and slow inference limit practical clinical deployment. Convolutional Neural Networks (CNNs) are more efficient, yet their locality restricts the modeling of long-range spatial dependencies that are important for complex anatomical structures. Recently, shift-based Multi-Layer Perceptrons (MLPs) have emerged as efficient alternatives for global feature interaction without self-attention, but most existing methods rely on fixed shift patterns and lack adaptability to spatial and frequency variations commonly observed in medical images, especially around edges and textured regions. To address these limitations, we propose a frequency-aware architectural refinement framework that combines frequency-guided feature decomposition with a soft, adaptive, and data-driven directional shift mechanism within a UNet-style backbone. Instead of using rigid shift rules, the method dynamically adjusts spatial displacements based on the frequency content of features, enabling more flexible interactions that enhance structural consistency and boundary sensitivity. Extensive experiments on four benchmark medical image segmentation datasets show that the proposed framework achieves consistently strong performance compared with representative CNN-, Transformer-, MLP-, and Mamba-based models, while maintaining favorable computational efficiency. Code is available at https://github.com/guangguangLi/FMLP.