TY - JOUR
T1 - A precise method for RBMs training using phased curricula
AU - Wang, Qianglong
AU - Gao, Xiaoguang
AU - Li, Xinyu
AU - Hu, Zijian
AU - Wan, Kaifang
N1 - Publisher Copyright:
© 2022, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.
PY - 2023/3
Y1 - 2023/3
N2 - Restricted Boltzmann machines (RBMs) are efficacious undirected neural networks for generating features and reconstructing images. Nevertheless, the classical persistent chain sampling algorithm has the problem of refactoring failure in the early training stage, which significantly limits the feature extraction and application of RBM. In this paper, motivated by the cumulative nature of the curriculum learning, three Phased Gibbs Sampling (PGS) methods are proposed for more efficient feature extraction and reconstruction by training the RBM periodically. Then, to achieve an automatic and exclusive training step, the innovative Improved Dynamic Learning Rate (IDLR) is designed by cooperating with the reconstruction error and the anti-vibration coefficient. Extensive experimental results of MNIST, 20 Newsgroup, Olivetti face, MNORB, and USPS demonstrate the superiority of three PGS-IDLR algorithms in terms of reconstruction error, training time, and classification accuracy. More specifically, the proposed algorithms can improve the classification accuracy by at least 2% and shorten the training time, compared with the state-of-the-art approaches. Moreover, they achieve a better performance in log-likelihood indictor and image reconstruction.
AB - Restricted Boltzmann machines (RBMs) are efficacious undirected neural networks for generating features and reconstructing images. Nevertheless, the classical persistent chain sampling algorithm has the problem of refactoring failure in the early training stage, which significantly limits the feature extraction and application of RBM. In this paper, motivated by the cumulative nature of the curriculum learning, three Phased Gibbs Sampling (PGS) methods are proposed for more efficient feature extraction and reconstruction by training the RBM periodically. Then, to achieve an automatic and exclusive training step, the innovative Improved Dynamic Learning Rate (IDLR) is designed by cooperating with the reconstruction error and the anti-vibration coefficient. Extensive experimental results of MNIST, 20 Newsgroup, Olivetti face, MNORB, and USPS demonstrate the superiority of three PGS-IDLR algorithms in terms of reconstruction error, training time, and classification accuracy. More specifically, the proposed algorithms can improve the classification accuracy by at least 2% and shorten the training time, compared with the state-of-the-art approaches. Moreover, they achieve a better performance in log-likelihood indictor and image reconstruction.
KW - Deep learning
KW - Improved dynamic learning rate
KW - Phased curricula
KW - Restricted boltzmann machines
UR - http://www.scopus.com/inward/record.url?scp=85129535584&partnerID=8YFLogxK
U2 - 10.1007/s11042-022-12973-2
DO - 10.1007/s11042-022-12973-2
M3 - 文章
AN - SCOPUS:85129535584
SN - 1380-7501
VL - 82
SP - 8013
EP - 8047
JO - Multimedia Tools and Applications
JF - Multimedia Tools and Applications
IS - 6
ER -