JOCSR: Joint optimization of camera spectral sensitivity and spectral reconstruction

Recovering hyperspectral images (HSIs) from low-dimensional spectral measurements (e.g., RGB or multispectral images) presents an efficient alternative to direct hyperspectral acquisition. While recent studies demonstrate that optimizing the camera spectral sensitivity (CSS) can markedly improve spectral reconstruction (SR) accuracy, prevailing methods typically rely on exhaustive search strategies, which are often inefficient and yield suboptimal results. To overcome these limitations, we propose JOCSR, a novel framework for the joint optimization of CSS and SR across various input dimensions. Our approach incorporates a Band Mapping Network (BMNet) to extract spectral features from hyperspectral data and perform one-dimensional mapping for CSS optimization. Multiple BMNet modules are interconnected with the SR network through a physical generative model for end-to-end training. Furthermore, we introduce a Channel-Attentive Spectral Fusion (CASF) module to enhance the SR model's representational capacity. From a practical standpoint, we devise a polynomial-based joint loss function that not only guides the network training but also incorporates fabrication constraints for the CSS. Extensive experiments on the ARAD 1 K and CAVE datasets show that JOCSR significantly surpasses state-of-the-art methods, achieving over 25% improvement in SR performance metrics. Notably, we designed and fabricated optical filters based on the optimized CSS, assembling a multispectral camera that validates our SR model with excellent results. This work demonstrates that flexibly optimizing CSS for arbitrary channel configurations not only boosts SR accuracy but also facilitates the development of compact and cost-effective hyperspectral imaging systems. The source code is available at https://github.com/tgg-77/JOCSR.