Toward High-Quality HDR Deghosting with Conditional Diffusion Models

Qingsen Yan, Tao Hu, Yuan Sun, Hao Tang, Yu Zhu, Wei Dong, Luc Van Gool, Yanning Zhang

Research output: Contribution to journalArticlepeer-review

40 Scopus citations

Abstract

High Dynamic Range (HDR) images can be recovered from several Low Dynamic Range (LDR) images by existing Deep Neural Networks (DNNs) techniques. Despite the remarkable progress, DNN-based methods still generate ghosting artifacts when LDR images have saturation and large motion, which hinders potential applications in real-world scenarios. To address this challenge, we formulate the HDR deghosting problem as an image generation that leverages LDR features as the diffusion model's condition, consisting of the feature condition generator and the noise predictor. Feature condition generator employs attention and Domain Feature Alignment (DFA) layer to transform the intermediate features to avoid ghosting artifacts. With the learned features as conditions, the noise predictor leverages a stochastic iterative denoising process for diffusion models to generate an HDR image by steering the sampling process. Furthermore, to mitigate semantic confusion caused by the saturation problem of LDR images, we design a sliding window noise estimator to sample smooth noise in a patch-based manner. In addition, an image space loss is proposed to avoid the color distortion of the estimated HDR results. We empirically evaluate our model on benchmark datasets for HDR imaging. The results demonstrate that our approach achieves state-of-the-art performances and well generalization to real-world images.

Original languageEnglish
Pages (from-to)4011-4026
Number of pages16
JournalIEEE Transactions on Circuits and Systems for Video Technology
Volume34
Issue number5
DOIs
StatePublished - 1 May 2024

Keywords

  • High dynamic range image
  • diffusion model
  • ghosting artifacts
  • multi-exposed imaging

Fingerprint

Dive into the research topics of 'Toward High-Quality HDR Deghosting with Conditional Diffusion Models'. Together they form a unique fingerprint.

Cite this