An information theoretic kernel algorithm for robust online learning

Haijin Fan, Qing Song, Zhao Xu

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations

Abstract

Kernel methods are widely used in nonlinear modeling applications. In this paper, a robust information theoretic sparse kernel algorithm is proposed for online learning. In order to reduce the computational cost and make the algorithm suitable for online applications, we investigate an information theoretic sparsification rule based on the mutual information between the system input and output to determine the update of the dictionary (support vectors). According to the rule, only novel and informative samples are selected to form a sparse and compact dictionary. Furthermore, to improve the generalization ability, a robust learning scheme is proposed to avoid the algorithm over learning the redundant samples, which assures the convergence of the learning algorithm and makes the learning algorithm converge to its steady state much faster. Experiment are conducted on practical and simulated data and results are shown to validate the effectiveness of our proposed algorithm.

Original languageEnglish
Title of host publication2012 International Joint Conference on Neural Networks, IJCNN 2012
DOIs
StatePublished - 2012
Externally publishedYes
Event2012 Annual International Joint Conference on Neural Networks, IJCNN 2012, Part of the 2012 IEEE World Congress on Computational Intelligence, WCCI 2012 - Brisbane, QLD, Australia
Duration: 10 Jun 201215 Jun 2012

Publication series

NameProceedings of the International Joint Conference on Neural Networks

Conference

Conference2012 Annual International Joint Conference on Neural Networks, IJCNN 2012, Part of the 2012 IEEE World Congress on Computational Intelligence, WCCI 2012
Country/TerritoryAustralia
CityBrisbane, QLD
Period10/06/1215/06/12

Keywords

  • dead zone
  • instantaneous mutual information
  • kernel algorithm
  • robust learning

Fingerprint

Dive into the research topics of 'An information theoretic kernel algorithm for robust online learning'. Together they form a unique fingerprint.

Cite this