TY - JOUR
T1 - A Deep Joint Network for Multispectral Demosaicking Based on Pseudo-Panchromatic Images
AU - Liu, Shumin
AU - Zhang, Yuge
AU - Chen, Jie
AU - Lim, Keng Pang
AU - Rahardja, Susanto
N1 - Publisher Copyright:
© 2007-2012 IEEE.
PY - 2022/6/1
Y1 - 2022/6/1
N2 - Single-sensor multispectral cameras generally utilize a multispectral filter array (MSFA) to sample spatial-spectral information for a reduced capturing time. However, in this situation, each pixel in an MSFA image only contains information from a single channel. Thus, demosaicking is necessary to reconstruct a full-resolution multispectral image from the raw MSFA image. In this paper, we propose a novel end-to-end deep learning framework based on pseudo-panchromatic images (PPIs), which consists of two networks, namely the Deep PPI Generation Network (DPG-Net) and Deep Demosaic Network (DDM-Net). Among them, we first pre-train DPG-Net to reconstruct a full-resolution panchromatic image from the raw MSFA image and then jointly train both networks to recover a full-resolution multispectral image, followed by fine-tuning both networks with fewer restrictions. Experimental results reveal that the proposed method outperforms state-of-the-art traditional and deep learning demosaicking methods both qualitatively and quantitatively.
AB - Single-sensor multispectral cameras generally utilize a multispectral filter array (MSFA) to sample spatial-spectral information for a reduced capturing time. However, in this situation, each pixel in an MSFA image only contains information from a single channel. Thus, demosaicking is necessary to reconstruct a full-resolution multispectral image from the raw MSFA image. In this paper, we propose a novel end-to-end deep learning framework based on pseudo-panchromatic images (PPIs), which consists of two networks, namely the Deep PPI Generation Network (DPG-Net) and Deep Demosaic Network (DDM-Net). Among them, we first pre-train DPG-Net to reconstruct a full-resolution panchromatic image from the raw MSFA image and then jointly train both networks to recover a full-resolution multispectral image, followed by fine-tuning both networks with fewer restrictions. Experimental results reveal that the proposed method outperforms state-of-the-art traditional and deep learning demosaicking methods both qualitatively and quantitatively.
KW - Demosaicking
KW - end-to-end deep learning
KW - multispectral filter array
KW - pseudo-panchromatic image
UR - http://www.scopus.com/inward/record.url?scp=85132510606&partnerID=8YFLogxK
U2 - 10.1109/JSTSP.2022.3172865
DO - 10.1109/JSTSP.2022.3172865
M3 - 文章
AN - SCOPUS:85132510606
SN - 1932-4553
VL - 16
SP - 622
EP - 635
JO - IEEE Journal on Selected Topics in Signal Processing
JF - IEEE Journal on Selected Topics in Signal Processing
IS - 4
ER -