TY - JOUR
T1 - Two-Stream Encoder GAN with Progressive Training for Co-Saliency Detection
AU - Qian, Xiaoliang
AU - Cheng, Xi
AU - Cheng, Gong
AU - Yao, Xiwen
AU - Jiang, Liying
N1 - Publisher Copyright:
© 1994-2012 IEEE.
PY - 2021
Y1 - 2021
N2 - The recent end-to-end co-saliency models have good performance, however, they cannot express the semantic consistency among a group of images well and usually require many co-saliency labels. To this end, a two-stream encoder generative adversarial network (TSE-GAN) with progressive training is proposed in this paper. In the pre-training stage, the salient object detection generative adversarial networks (SOD-GAN) and classification network (CN) are separately trained by the salient object detection (SOD) datasets and co-saliency datasets with only category labels to learn the intra-saliency and preliminary inter-saliency cues and alleviate the problem of insufficient co-saliency labels. In the second training stage, the backbone of TSE-GAN is inherited from the trained SOD-GAN, the encoder of trained SOD-GAN (SOD-Encoder) is used to extract intra-saliency features, the group-wise semantic encoder (GS-Encoder) is constructed by the multi-level group-wise category features extracted from CN for extracting inter-saliency features with better semantic consistency, the TSE-GAN constructed by incorporating the GS-Encoder into SOD-GAN is trained on co-saliency datasets for co-saliency detection. The comprehensive comparisons with 13 state-of-the-art methods demonstrate the effectiveness of proposed method.
AB - The recent end-to-end co-saliency models have good performance, however, they cannot express the semantic consistency among a group of images well and usually require many co-saliency labels. To this end, a two-stream encoder generative adversarial network (TSE-GAN) with progressive training is proposed in this paper. In the pre-training stage, the salient object detection generative adversarial networks (SOD-GAN) and classification network (CN) are separately trained by the salient object detection (SOD) datasets and co-saliency datasets with only category labels to learn the intra-saliency and preliminary inter-saliency cues and alleviate the problem of insufficient co-saliency labels. In the second training stage, the backbone of TSE-GAN is inherited from the trained SOD-GAN, the encoder of trained SOD-GAN (SOD-Encoder) is used to extract intra-saliency features, the group-wise semantic encoder (GS-Encoder) is constructed by the multi-level group-wise category features extracted from CN for extracting inter-saliency features with better semantic consistency, the TSE-GAN constructed by incorporating the GS-Encoder into SOD-GAN is trained on co-saliency datasets for co-saliency detection. The comprehensive comparisons with 13 state-of-the-art methods demonstrate the effectiveness of proposed method.
KW - Co-saliency detection
KW - progressive training
KW - two-stream encoders
UR - http://www.scopus.com/inward/record.url?scp=85099578829&partnerID=8YFLogxK
U2 - 10.1109/LSP.2021.3049997
DO - 10.1109/LSP.2021.3049997
M3 - 文章
AN - SCOPUS:85099578829
SN - 1070-9908
VL - 28
SP - 180
EP - 184
JO - IEEE Signal Processing Letters
JF - IEEE Signal Processing Letters
M1 - 9316738
ER -