TY - GEN
T1 - An information theoretic kernel algorithm for robust online learning
AU - Fan, Haijin
AU - Song, Qing
AU - Xu, Zhao
PY - 2012
Y1 - 2012
N2 - Kernel methods are widely used in nonlinear modeling applications. In this paper, a robust information theoretic sparse kernel algorithm is proposed for online learning. In order to reduce the computational cost and make the algorithm suitable for online applications, we investigate an information theoretic sparsification rule based on the mutual information between the system input and output to determine the update of the dictionary (support vectors). According to the rule, only novel and informative samples are selected to form a sparse and compact dictionary. Furthermore, to improve the generalization ability, a robust learning scheme is proposed to avoid the algorithm over learning the redundant samples, which assures the convergence of the learning algorithm and makes the learning algorithm converge to its steady state much faster. Experiment are conducted on practical and simulated data and results are shown to validate the effectiveness of our proposed algorithm.
AB - Kernel methods are widely used in nonlinear modeling applications. In this paper, a robust information theoretic sparse kernel algorithm is proposed for online learning. In order to reduce the computational cost and make the algorithm suitable for online applications, we investigate an information theoretic sparsification rule based on the mutual information between the system input and output to determine the update of the dictionary (support vectors). According to the rule, only novel and informative samples are selected to form a sparse and compact dictionary. Furthermore, to improve the generalization ability, a robust learning scheme is proposed to avoid the algorithm over learning the redundant samples, which assures the convergence of the learning algorithm and makes the learning algorithm converge to its steady state much faster. Experiment are conducted on practical and simulated data and results are shown to validate the effectiveness of our proposed algorithm.
KW - dead zone
KW - instantaneous mutual information
KW - kernel algorithm
KW - robust learning
UR - http://www.scopus.com/inward/record.url?scp=84865084106&partnerID=8YFLogxK
U2 - 10.1109/IJCNN.2012.6252837
DO - 10.1109/IJCNN.2012.6252837
M3 - 会议稿件
AN - SCOPUS:84865084106
SN - 9781467314909
T3 - Proceedings of the International Joint Conference on Neural Networks
BT - 2012 International Joint Conference on Neural Networks, IJCNN 2012
T2 - 2012 Annual International Joint Conference on Neural Networks, IJCNN 2012, Part of the 2012 IEEE World Congress on Computational Intelligence, WCCI 2012
Y2 - 10 June 2012 through 15 June 2012
ER -