Insights Into the Robustness of Minimum Error Entropy Estimation

Badong Chen, Lei Xing, Bin Xu, Haiquan Zhao, José C. Príncipe

Research output: Contribution to journalArticlepeer-review

67 Scopus citations

Abstract

The minimum error entropy (MEE) is an important and highly effective optimization criterion in information theoretic learning (ITL). For regression problems, MEE aims at minimizing the entropy of the prediction error such that the estimated model preserves the information of the data generating system as much as possible. In many real world applications, the MEE estimator can outperform significantly the well-known minimum mean square error (MMSE) estimator and show strong robustness to noises especially when data are contaminated by non-Gaussian (multimodal, heavy tailed, discrete valued, and so on) noises. In this brief, we present some theoretical results on the robustness of MEE. For a one-parameter linear errors-in-variables (EIV) model and under some conditions, we derive a region that contains the MEE solution, which suggests that the MEE estimate can be very close to the true value of the unknown parameter even in presence of arbitrarily large outliers in both input and output variables. Theoretical prediction is verified by an illustrative example.

Original languageEnglish
Pages (from-to)731-737
Number of pages7
JournalIEEE Transactions on Neural Networks and Learning Systems
Volume29
Issue number3
DOIs
StatePublished - Mar 2018

Keywords

  • Estimation
  • minimum error entropy (MEE)
  • robustness

Fingerprint

Dive into the research topics of 'Insights Into the Robustness of Minimum Error Entropy Estimation'. Together they form a unique fingerprint.

Cite this