A precise method for RBMs training using phased curricula

Restricted Boltzmann machines (RBMs) are efficacious undirected neural networks for generating features and reconstructing images. Nevertheless, the classical persistent chain sampling algorithm has the problem of refactoring failure in the early training stage, which significantly limits the feature extraction and application of RBM. In this paper, motivated by the cumulative nature of the curriculum learning, three Phased Gibbs Sampling (PGS) methods are proposed for more efficient feature extraction and reconstruction by training the RBM periodically. Then, to achieve an automatic and exclusive training step, the innovative Improved Dynamic Learning Rate (IDLR) is designed by cooperating with the reconstruction error and the anti-vibration coefficient. Extensive experimental results of MNIST, 20 Newsgroup, Olivetti face, MNORB, and USPS demonstrate the superiority of three PGS-IDLR algorithms in terms of reconstruction error, training time, and classification accuracy. More specifically, the proposed algorithms can improve the classification accuracy by at least 2% and shorten the training time, compared with the state-of-the-art approaches. Moreover, they achieve a better performance in log-likelihood indictor and image reconstruction.