TY - JOUR
T1 - Image deraining via invertible disentangled representations
AU - Chen, Xueling
AU - Zhou, Xuan
AU - Sun, Wei
AU - Zhang, Yanning
N1 - Publisher Copyright:
© 2024
PY - 2024/11
Y1 - 2024/11
N2 - Photos taken on rainy days usually suffer from a loss of background image information due to obstruction by rain streaks and the mist-like effect that rain creates. Recent Invertible Neural Networks (INNs) have shown promise in image deraining, as they are capable of extracting observed features without information loss for the purpose of restoration. However, it remains a challenge for INNs to directly learn the transformation from a rainy image to its rain-free version, given that rain streaks exhibit a diversity in transparency, shape, and brightness, leading to a complex distribution of lost details. To solve the problem, we design a novel Invertible Disentangling network (InvDis) to decompose the complex data distribution into simpler ones and makes the transformation much easier to learning. To be specific, InvDis separates feature channels of rain streaks and those of the polluted image details in the forward mapping. By only discarding rain channels and preserving polluted image details, in the backward mapping, InvDis focuses on generating missing details obstructed by rain streaks with a new sample from a prior distribution, and restores clean details from those polluted ones. Additionally, InvDis incorporates a channel interaction mechanism to facilitate the disentangling. It allows the rain streaks encoded in other channels to easily flow to those discarded channels, and conversely, allows the polluted image details to be directed towards the preserved channels. Experimental results on both synthetic and real-world datasets show that the InvDis not only improves the restoration quality but also has lower computational costs.
AB - Photos taken on rainy days usually suffer from a loss of background image information due to obstruction by rain streaks and the mist-like effect that rain creates. Recent Invertible Neural Networks (INNs) have shown promise in image deraining, as they are capable of extracting observed features without information loss for the purpose of restoration. However, it remains a challenge for INNs to directly learn the transformation from a rainy image to its rain-free version, given that rain streaks exhibit a diversity in transparency, shape, and brightness, leading to a complex distribution of lost details. To solve the problem, we design a novel Invertible Disentangling network (InvDis) to decompose the complex data distribution into simpler ones and makes the transformation much easier to learning. To be specific, InvDis separates feature channels of rain streaks and those of the polluted image details in the forward mapping. By only discarding rain channels and preserving polluted image details, in the backward mapping, InvDis focuses on generating missing details obstructed by rain streaks with a new sample from a prior distribution, and restores clean details from those polluted ones. Additionally, InvDis incorporates a channel interaction mechanism to facilitate the disentangling. It allows the rain streaks encoded in other channels to easily flow to those discarded channels, and conversely, allows the polluted image details to be directed towards the preserved channels. Experimental results on both synthetic and real-world datasets show that the InvDis not only improves the restoration quality but also has lower computational costs.
KW - Disentangled representation
KW - Image deraining
KW - Invertible channel interaction mechanism
KW - Invertible neural networks
UR - http://www.scopus.com/inward/record.url?scp=85202556652&partnerID=8YFLogxK
U2 - 10.1016/j.engappai.2024.109207
DO - 10.1016/j.engappai.2024.109207
M3 - 文章
AN - SCOPUS:85202556652
SN - 0952-1976
VL - 137
JO - Engineering Applications of Artificial Intelligence
JF - Engineering Applications of Artificial Intelligence
M1 - 109207
ER -