SSST-GAN: A Sampling-Based Spatial-Spectral Transformer and Generative Adversarial Network for Hyperspectral Unmixing

  • Yu Zhang
  • , Jiageng Huang
  • , Yefei Huang
  • , Wei Gao
  • , Jie Chen

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

Transformer-based architectures have shown strong potential in hyperspectral unmixing due to their powerful modeling capabilities. However, most existing Transformer-based methods still struggle to effectively capture and fuse spatial-spectral features, and their predominant reliance on reconstruction error further constrains overall unmixing performance. Moreover, they rarely account for the nonlinear correlations that inherently exist between the spatial and spectral domains. To address these challenges, we propose a sampling-based spatial-spectral Transformer and generative adversarial network (SSST-GAN). The proposed model employs a dual-branch, sampling-based Transformer encoder to independently extract spatial and spectral representations. Specifically, the spatial branch adopts a full-sampling multi-head attention mechanism to capture rich contextual dependencies among spatial pixels, while the spectral branch utilizes a sparse sampling strategy to efficiently distill key information from high-dimensional spectral data. A feature enhancement module (FEM) is introduced to integrate and strengthen the complementary characteristics of spatial and spectral features. To further improve the modeling of complex nonlinear mixing patterns, we incorporate a generalized nonlinear fluctuation model (GNFM) at the decoding stage. In addition, SSST-GAN leverages a generative adversarial learning framework, in which a discriminator evaluates the authenticity of reconstructed pixels, thereby enhancing the fidelity of the unmixing results. Extensive experiments on both synthetic and real-world datasets demonstrate that SSST-GAN consistently outperforms several state-of-the-art methods in terms of unmixing accuracy.

Keywords

  • Attention
  • deep learning
  • generative adversarial network (GAN)
  • hyperspectral unmixing
  • spatial-spectral model
  • vision transformer

Fingerprint

Dive into the research topics of 'SSST-GAN: A Sampling-Based Spatial-Spectral Transformer and Generative Adversarial Network for Hyperspectral Unmixing'. Together they form a unique fingerprint.

Cite this