A Deep Joint Network for Multispectral Demosaicking Based on Pseudo-Panchromatic Images

Shumin Liu, Yuge Zhang, Jie Chen, Keng Pang Lim, Susanto Rahardja

科研成果: 期刊稿件文章同行评审

21 引用 (Scopus)

摘要

Single-sensor multispectral cameras generally utilize a multispectral filter array (MSFA) to sample spatial-spectral information for a reduced capturing time. However, in this situation, each pixel in an MSFA image only contains information from a single channel. Thus, demosaicking is necessary to reconstruct a full-resolution multispectral image from the raw MSFA image. In this paper, we propose a novel end-to-end deep learning framework based on pseudo-panchromatic images (PPIs), which consists of two networks, namely the Deep PPI Generation Network (DPG-Net) and Deep Demosaic Network (DDM-Net). Among them, we first pre-train DPG-Net to reconstruct a full-resolution panchromatic image from the raw MSFA image and then jointly train both networks to recover a full-resolution multispectral image, followed by fine-tuning both networks with fewer restrictions. Experimental results reveal that the proposed method outperforms state-of-the-art traditional and deep learning demosaicking methods both qualitatively and quantitatively.

源语言英语
页(从-至)622-635
页数14
期刊IEEE Journal on Selected Topics in Signal Processing
16
4
DOI
出版状态已出版 - 1 6月 2022

指纹

探究 'A Deep Joint Network for Multispectral Demosaicking Based on Pseudo-Panchromatic Images' 的科研主题。它们共同构成独一无二的指纹。

引用此