HC-XLSTM: A Dual-Branch Framework With Paired Blocks for Hyperspectral Image Classification

Hyperspectral image (HSI) classification requires learning rich spatial-spectral representations over hundreds of contiguous spectral bands. A key challenge is to capture both fine local spatial–spectral structures and long-range contextual dependencies efficiently. CNN-based methods are effective in modeling local patterns but are often limited by their receptive fields when global context is required. Transformer-based methods can capture long-range dependencies, yet their attention mechanism typically incurs quadratic complexity. These limitations motivate a linear-time model that jointly captures local details and long-range context. In this paper, we propose HCxLSTM, a dual-branch framework built on extended Long Short-Term Memory (xLSTM) for hyperspectral image classification. Our design employs Paired Blocks to process patch tokens in alternating directions, enabling a multi-directional scanning that preserves comprehensive spatial cues. One branch leverages mLSTM (matrix-based memory) for richer feature capacity and parallelizable state updates, while the other utilizes sLSTM (scalar-based memory) for fine-grained gating and frequent revisions. We further introduce a CrossmLSTM module that fuses upper-branch “queries” with lower-branch “key–value” features in a linear-time fashion. Through this two-branch design, HCxLSTM captures both long-range contextual relationships and localized spatial–spectral details. Extensive experiments on four HSI datasets demonstrate that our method outperforms state-of-the-art approaches in accuracy and efficiency.