HSPTrack: Hyperspectral Sequence Prediction Tracker with Transformers

Hyperspectral object tracking focuses on fully exploiting the spectral information of the object and the spectral characteristics of background to optimize tracking performance. However, the existing tracking methods usually employ the complete hyperspectral cube as input, which is computationally demanding and overlooks the incorporation of temporal information. In this paper, an end-to-end hyperspectral object tracker, named HSPTrack, is proposed to address these problems. The framework integrates a sequence prediction module, rooted in the principles of causal transformers, to seamlessly integrate temporal information. This integration is vital for maintaining effective and robust cross-frame tracking. The aspiration is for this paper to serve as a benchmark in constructing a universal model devoted to achieving highprecision hyperspectral object tracker. The performance of the framework is evaluated through its application to a publicly accessible hyperspectral video dataset containing 16 bands, 25 bands, and 15 bands. Quantitative experiments are conducted on a close-up hyperspectral video dataset of different bands, and verified that the proposed method achieves promising tracking performances, compared with the other state-of-the-art trackers.