Training restricted boltzmann machine using gradient fixing based algorithm

Fei Li, Xiaoguang Gao, Kaifang Wan

科研成果: 期刊稿件文章同行评审

7 引用 (Scopus)

摘要

Most of the algorithms for training restricted Boltzmann machines (RBM) are based on Gibbs sampling. When the sampling algorithm is used to calculate the gradient, the sampling gradient is the approximate value of the true gradient and there is a big error between the sampling gradient and the true gradient, which seriously affects the training effect of the network. Aiming at this problem, this paper analysed the numerical error and orientation error between the approximate gradient and the true gradient. Their influence on the performance of network training is given then. An gradient fixing model was established. It was designed to adjust the numerical value and orientation of the approximate gradient and reduce the error. We also designed gradient fixing based Gibbs sampling training algorithm (GFGS) and gradient fixing based parallel tempering algorithm (GFPT), and the comparison experiment of the novel algorithms and the existing algorithms is given. It has been demonstrated that the new algorithms can effectively tackle the issue of gradient error, and can achieve higher training accuracy at a reasonable expense of computational runtime.

源语言英语
页(从-至)694-703
页数10
期刊Chinese Journal of Electronics
27
4
DOI
出版状态已出版 - 10 7月 2018

指纹

探究 'Training restricted boltzmann machine using gradient fixing based algorithm' 的科研主题。它们共同构成独一无二的指纹。

引用此