Modeling with prejudice: Small-sample learning via adversary for semantic segmentation

Semantic segmentation has become one of the core tasks for scene understanding and many high-level works heavily rely on its performance. In the past decades, much progress has been achieved. However, some problems still need to be settled. One problem is about the challenging classification of various objects, which are with diverse viewpoints, illumination, appearance, and cluttered backgrounds, in a unified framework. The other one is focusing on the unbalanced distribution of semantic labels, where long-tail phenomenon exists and the trained model tends to be biased toward the majority classes when testing. And this problem can be regarded as the small-sample learning problem in semantic segmentation for the number of training samples upon the minority classes are small. For tackling these problems, a small-sample learning method via adversary is proposed and three contributions are claimed: 1) discriminatory modeling for semantic segmentation: Two submodels are simultaneously built based on the attribute of semantic classs; 2) hierarchical contextual information consideration: Both local and global contextual relationships are equally modeled under a hierarchical probabilistic graphical method and neighborhood relationship in label space are also considered; and 3) adversary learning for small-sample modeling: According to the structural relationships between small samples and the others, semantic classes are adversely modeled through computing the weighted costs. Experimental results on three benchmarks have verified the superiority of the proposed method compared with the state-of-the-arts.