Cascaded transformer U-net for image restoration

Longbin Yan, Min Zhao, Shumin Liu, Shuaikai Shi, Jie Chen

科研成果: 期刊稿件文章同行评审

25 引用 (Scopus)

摘要

Image restoration is one of the most important computer vision tasks, aiming at recovering high-quality images from degraded or low-quality observations. The restoration methods based on convolutional neural networks (CNNs) have achieved attractive performance, however, as convolutions only intake local information, CNN-based methods have limitations in modeling objects in long ranges and extracting global information. In addition, existing one-stage methods damage the performance due to lacking diversified receptive fields. In this paper, we propose a multi-stage cascaded transformer architecture for image restoration. Firstly, the Swin transformer based encoder relying on self-attention is used to improve the modeling ability for long-range objects and outputs hierarchical multi-level semantic features. Then, a shape perceiving module is designed and embedded in the decoder to enhance the representation of irregular objects, Moreover, a multi-stage cascaded encoder-decoder architecture possessing diversified receptive fields is proposed to progressively obtain fine restoration results and thus boost the performance. We conduct extensive experiments, including image deraining, underwater image enhancement, near infrared image colorization and low-light image enhancement. The results show that our proposed method can achieve comparable or better performance than state-of-the-art methods while with less training and inference costs.

源语言英语
文章编号108902
期刊Signal Processing
206
DOI
出版状态已出版 - 5月 2023

指纹

探究 'Cascaded transformer U-net for image restoration' 的科研主题。它们共同构成独一无二的指纹。

引用此