A Multi-Model Fusion Framework for NIR-to-RGB Translation

Longbin Yan, Xiuheng Wang, Min Zhao, Shumin Liu, Jie Chen

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

17 Scopus citations

Abstract

Near-infrared (NIR) images provide spectral information beyond the visible light spectrum and thus are useful in many applications. However, single-channel NIR images contain less information per pixel than RGB images and lack visibility for human perception. Transforming NIR images to RGB images is necessary for performing further analysis and computer vision tasks. In this work, we propose a novel NIR-to-RGB translation method. It contains two sub-networks and a fusion operator. Specifically, a U-net based neural network is used to learn the texture information while a CycleGAN based neural network is adopted to excavate the color information. Finally, a guided filter based fusion strategy is applied to fuse the outputs of these two neural networks. Experiment results show that our proposed method achieves superior NIR-to-RGB translation performance.

Original languageEnglish
Title of host publication2020 IEEE International Conference on Visual Communications and Image Processing, VCIP 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages459-462
Number of pages4
ISBN (Electronic)9781728180670
DOIs
StatePublished - 1 Dec 2020
Event2020 IEEE International Conference on Visual Communications and Image Processing, VCIP 2020 - Virtual, Macau, China
Duration: 1 Dec 20204 Dec 2020

Publication series

Name2020 IEEE International Conference on Visual Communications and Image Processing, VCIP 2020

Conference

Conference2020 IEEE International Conference on Visual Communications and Image Processing, VCIP 2020
Country/TerritoryChina
CityVirtual, Macau
Period1/12/204/12/20

Keywords

  • CycleGAN
  • guided filter
  • image fusion
  • NIR-to-RGB translation
  • U-net

Fingerprint

Dive into the research topics of 'A Multi-Model Fusion Framework for NIR-to-RGB Translation'. Together they form a unique fingerprint.

Cite this