A Deep Joint Network for Multispectral Demosaicking Based on Pseudo-Panchromatic Images

Shumin Liu, Yuge Zhang, Jie Chen, Keng Pang Lim, Susanto Rahardja

Research output: Contribution to journalArticlepeer-review

21 Scopus citations

Abstract

Single-sensor multispectral cameras generally utilize a multispectral filter array (MSFA) to sample spatial-spectral information for a reduced capturing time. However, in this situation, each pixel in an MSFA image only contains information from a single channel. Thus, demosaicking is necessary to reconstruct a full-resolution multispectral image from the raw MSFA image. In this paper, we propose a novel end-to-end deep learning framework based on pseudo-panchromatic images (PPIs), which consists of two networks, namely the Deep PPI Generation Network (DPG-Net) and Deep Demosaic Network (DDM-Net). Among them, we first pre-train DPG-Net to reconstruct a full-resolution panchromatic image from the raw MSFA image and then jointly train both networks to recover a full-resolution multispectral image, followed by fine-tuning both networks with fewer restrictions. Experimental results reveal that the proposed method outperforms state-of-the-art traditional and deep learning demosaicking methods both qualitatively and quantitatively.

Original languageEnglish
Pages (from-to)622-635
Number of pages14
JournalIEEE Journal on Selected Topics in Signal Processing
Volume16
Issue number4
DOIs
StatePublished - 1 Jun 2022

Keywords

  • Demosaicking
  • end-to-end deep learning
  • multispectral filter array
  • pseudo-panchromatic image

Fingerprint

Dive into the research topics of 'A Deep Joint Network for Multispectral Demosaicking Based on Pseudo-Panchromatic Images'. Together they form a unique fingerprint.

Cite this