高光谱图像超分辨率重建技术研究进展

Computer vision-oriented hyperspectral images（HSIs）are featured to enrich spatial and spectral information compared to gray and RGB images. It has been developing in such domains like target detection，scene classification，and tracking. However，the HSI imaging technique is challenged for its distortion problems（e. g. ，low spatial resolution，noise）. HSI-related super-resolution（SR）methods are proposed to reconstruct high-quality HSIs in terms of a high spectral resolution and spatial resolution（HR）. Current HSI SR methods can be segmented into two categories：spatial SR and spectral SR. The spatial SR method is oriented to reconstruct the target HR HSI via improving the spatial resolution of low-resolution（LR）HSI. It can be subdivided into single image SR and fusion based SR methods further. Single image based SR method can be used to reconstruct the target HSI via directly improving the spatial resolution of LR HSI. However，due to much more spatial information is sacrificed，single image based SR method is challenged to reconstruct effective HSIs. Therefore，to fuse these high quality spatial information into the LR HSIs，extra high spatial homogeneous information is introduced（e. g. ，multispectral image（MSI），RGB）. In this way，the spatial resolution of LR HSI can be improved greatly（e. g. ，8 times，16 times，and 32 times SR）. The other HSI SR method is focused on the spectral super-resolution method，which can improve the spectral resolution of high spatial resolution images（e. g. ，MSI，RGB）and generate the target hyperspectral image. The literature review is focused on the growth of HSI SR methods in relevance to three aspects single image-based，fusion-based，and spectral super-resolution contexts. Each of these three category for HSI SR methods can be further subdivided into two aspects of traditional optimization framework and deep learning based methods. For single image HSI SR，due to the SR problem being an illness inverse problem，traditional optimization framework based single image HSI SR methods is used to develop effective image priors to restrain the SR process. Image priors like low-rank，sparse representation，and non-local features are commonly used in single based HSI SR method，but it still challenged for such problems of manpower and restrictions. For the traditional fusion based method，the core element is to balance the spatial-spectral correlation between HR MSI and LR HSI. It is feasible to split these two images into key components and then re-combine the effective parts of each image. Therefore，multiple schemes are introduced（e. g. ，nonnegative matrix factorization，coupled tensor factorization）to leverage the key information from HR MSI and LR HSI. In addition，to increase the effectiveness of these decomposition methods，some constraints are required to be introduced （e. g. ，sparse，low-rank）. For the traditional spectral resolution method，it is essential to learn how to reconstruct the spectral characteristics from RGB/MSI images. When paired RGB and HSI exist，a promising way is feasible to construct a dictionary（e. g. ，sparse dictionary learning）that the mapping relation can be recorded between RGB/MSI images and HSIs. The dictionary learning based spectral super-resolution methods have been developing as well，but it is often challenged to realize more generalization ability for applications. In recent years，deep learning based methods have facilitated much more computer vision tasks，and is beneficial for exploiting the inherent spatial-spectral relations of HSIs. For the single image HSI SR methods，a deep convolution neural network（DCNN）is utilized to learn the mapping process from LR HSI to HR HSI. The DCNN is capable to learn deep image prior from plenty of training samples，which has better representation ability than the heuristic handcrafted image priors（e. g. ，low-rank，sparse）to some extent. However，the performance of this kind of method is often restricted by the amount and variety of training samples. Therefore，it is necessary to reconstruct HR HSI from LR HSI in an unsupervised manner，but the robustness of the unsupervised method is still a challenging problem to be resolved. For the deep learning based HSI fusion methods，to extract spatial-spectral information from MSIs and HSIs，an effective design of DCNN frameworks（e. g. ，multi-branch，multi-scale，3D-CNN）are concerned about. However，such problems（e. g. ，noise，unknown degeneration，unregistered）are challenged for effective generalization ability of DCNN-based HSI fusion method. To resolve these problems，unsupervised and alternative learning methods are introduced to improve fusion-based generalization ability. The DCNN-unfolding is proposed and the optimal interpret-ability is demonstrated，and its registration strategy is introduced to improve the robustness further. For the spectral super resolution method，DCNN is demonstrated to model the mapping scheme between RGB/MSI and HR HSI. However，there still some barriers are required to be resolved for DCNN-based spectral super resolution method. For example，most existing spectral super resolution methods can generate HSI only according to its fixed spectral interval or spectrum range. A spectral super resolution framework is required to be linked with generalization ability well in the future. In this study，we will summarize the developing of the HSI SR scope from the perspective of new designs，new methods，and new application scenes.