Adversarial learning with collaborative attention for facial makeup removal

In this paper, we propose an adversarial learning method with collaborative attention to remove facial makeup in profile photos using generative adversarial networks (GANs). One challenge of makeup removal is how to preserve key characteristics, such as eye color, while thoroughly removing makeup. A collaborative attention mechanism is proposed to selectively manipulate face regions. In the mechanism, intra-attention takes the self-correlation of faces with makeup into consideration, which restricts inter-attention to pay more attention to regions with heavier makeup while leaving other regions untouched (e.g., eyes and hair). Another challenge is that the similarity between light makeup and non-makeup images results in a small domain gap. In such cases, it is hard to distinguish and thoroughly remove light makeup. To strengthen discriminative ability, we propose using input makeup images as negative samples to train a new-style classifier for the discriminator. The classifier enhances the discriminator to extract more discriminative style features and helps the generator to thoroughly remove light makeup. To validate the proposed algorithm, a large-scale makeup/non-makeup dataset was built. Extensive experiments demonstrated the effectiveness of the proposed method, achieving new state-of-the-art performance.