Generalized and Robust Least Squares Regression

Jingyu Wang, Fangyuan Xie, Feiping Nie, Xuelong Li

Research output: Contribution to journalArticlepeer-review

9 Scopus citations

Abstract

As a simple yet effective method, least squares regression (LSR) is extensively applied for data regression and classification. Combined with sparse representation, LSR can be extended to feature selection (FS) as well, in which l1 regularization is often applied in embedded FS algorithms. However, because the loss function is in the form of squared error, LSR and its variants are sensitive to noises, which significantly degrades the effectiveness and performance of classification and FS. To cope with the problem, we propose a generalized and robust LSR (GRLSR) for classification and FS, which is made up of arbitrary concave loss function and the l2, p-norm regularization term. Meanwhile, an iterative algorithm is applied to efficiently deal with the nonconvex minimization problem, in which an additional weight to suppress the effect of noises is added to each data point. The weights can be automatically assigned according to the error of the samples. When the error is large, the value of the corresponding weight is small. It is this mechanism that allows GRLSR to reduce the impact of noises and outliers. According to the different formulations of the concave loss function, four specific methods are proposed to clarify the essence of the framework. Comprehensive experiments on corrupted datasets have proven the advantage of the proposed method.

Original languageEnglish
Pages (from-to)7006-7020
Number of pages15
JournalIEEE Transactions on Neural Networks and Learning Systems
Volume35
Issue number5
DOIs
StatePublished - 1 May 2024

Keywords

  • Classification
  • feature selection (FS)
  • generalized framework
  • linear regression
  • robust

Fingerprint

Dive into the research topics of 'Generalized and Robust Least Squares Regression'. Together they form a unique fingerprint.

Cite this