Localized Multiple Kernel Learning With Dynamical Clustering and Matrix Regularization

Yina Han, Kunde Yang, Yixin Yang, Yuanliang Ma

Research output: Contribution to journalArticlepeer-review

35 Scopus citations

Abstract

Localized multiple kernel learning (LMKL) is an attractive strategy for combining multiple heterogeneous features with regard to their discriminative power for each individual sample. However, the learning of numerous local solutions may not scale well even for a moderately sized training set, and the independently learned local models may suffer from overfitting. Hence, in existing local methods, the distributed samples are typically assumed to share the same weights, and various unsupervised clustering methods are applied as preprocessing. In this paper, to enable the learner to discover and benefit from the underlying local coherence and diversity of the samples, we incorporate the clustering procedure into the canonical support vector machine-based LMKL framework. Then, to explore the relatedness among different samples, which has been ignored in a vector ℓp-norm analysis, we organize the cluster-specific kernel weights into a matrix and introduce a matrix-based extension of the ℓp-norm for constraint enforcement. By casting the joint optimization problem as a problem of alternating optimization, we show how the cluster structure is gradually revealed and how the matrix-regularized kernel weights are obtained. A theoretical analysis of such a regularizer is performed using a Rademacher complexity bound, and complementary empirical experiments on real-world data sets demonstrate the effectiveness of our technique.

Original languageEnglish
Article number7792117
Pages (from-to)486-499
Number of pages14
JournalIEEE Transactions on Neural Networks and Learning Systems
Volume29
Issue number2
DOIs
StatePublished - Feb 2018

Keywords

  • Dynamical clustering
  • localized multiple kernel learning (LMKL)
  • matrix regularization
  • support vector machine (SVM)

Fingerprint

Dive into the research topics of 'Localized Multiple Kernel Learning With Dynamical Clustering and Matrix Regularization'. Together they form a unique fingerprint.

Cite this