Accelerated learning for restricted boltzmann machine with a novel momentum algorithm

Xiaoguang Gao, Fei Li, Kaifang Wan

Research output: Contribution to journalArticlepeer-review

6 Scopus citations

Abstract

We investigated two commonly used momentum algorithms, Classical momentum (CM) and Nesterov momentum (NM). We found that, when used in Restricted Boltzmann machine (RBM), they have two main problems: The first one is their performances are not obvious and not as good as expected. The second one is they may lose accelerating ability in the later stage of training process. Aiming at these two problems, we proposed the Weight momentum algorithm and evaluated our approach on four datasets. It has been demonstrated that our methods can achieve better performance under both reconstruction error and classification rate criterions.

Original languageEnglish
Pages (from-to)483-487
Number of pages5
JournalChinese Journal of Electronics
Volume27
Issue number3
DOIs
StatePublished - 10 May 2018

Keywords

  • Deep learning
  • Momentum algorithm
  • Restricted Boltzmann ma chine (RBM)
  • Weight momentum (WM)

Fingerprint

Dive into the research topics of 'Accelerated learning for restricted boltzmann machine with a novel momentum algorithm'. Together they form a unique fingerprint.

Cite this