Deep multi-task learning for shadow detection and removal

Shadows actually play an important role in image understanding. But even for the same object, the intensity and shape of the shadows can vary with the environment. Thus it is a quite changeling issue to detect and remove shadows from images. Recent studies have been trying to solve these two tasks independently, but they are closely related to each other actually. Therefore, we propose a multi-task adversarial generative networks (mtGAN) that can detect and remove shadows simultaneously. For the proposed mtGAN, the cross-stitch unit is applied to learn the optimal ways to share features between multi-tasks, which is not set empirically as usual. Also, the combination weight of multi-task loss functions are trained according to the uncertainty distribution of each task. Based on these multi-task learning strategies, the proposed mtGAN can achieve shadow detection and removal tasks better than existing methods. In experiments, the effectiveness of the proposed mtGAN is shown.