TY - JOUR
T1 - Robust support vector ordinal regression
AU - Xiang, Haorui
AU - Wu, Zhichang
AU - Wang, Rong
AU - Nie, Feiping
AU - Li, Xuelong
N1 - Publisher Copyright:
© 2025 Elsevier Inc.
PY - 2025/11
Y1 - 2025/11
N2 - Ordinal regression is a unique supervised problem where the labels exhibit a natural order, setting it apart from multi-class classification and metric regression. While Support Vector Ordinal Regression with Explicit Constraints (SVOREX) is a widely used model for ordinal regression, it is highly sensitive to outliers in the training set. In many pattern recognition and machine learning tasks, outliers are often present in the training set. These outliers can mislead the learning process of the learner, leading to sub-optimal performance. In this paper, we propose a novel Graduated Escalation (GE) loss that uses a truncation strategy to handle outliers. The GE loss can help the model to detect and eliminate the outliers in the training process. Adhering to this concept, we present a new robust support vector ordinal regression (RSVOR) model that is robust to outliers. Existing robust support vector ordinal regression models, like robust classification models, often use a truncation function to ensure the model's robustness to outliers. However, this simple truncation function ignores the unique order relationship in ordinal regression. The Graduated Escalation function of RSVOR ensures robustness to outliers while also preserving the order information, a characteristic often overlooked by existing robust ordinal regression models. A binary weight matrix is used in RSVOR to identify and eliminate outliers to improve the robustness against outliers. Additionally, we develop a new optimization algorithm based on difference of convex (DC) algorithm to efficiently minimize the GE loss objective. Theoretical results demonstrate the convergence of our optimization algorithm. Extensive empirical results show that our method outperforms state-of-the-art ordinal regression methods on datasets containing outliers.
AB - Ordinal regression is a unique supervised problem where the labels exhibit a natural order, setting it apart from multi-class classification and metric regression. While Support Vector Ordinal Regression with Explicit Constraints (SVOREX) is a widely used model for ordinal regression, it is highly sensitive to outliers in the training set. In many pattern recognition and machine learning tasks, outliers are often present in the training set. These outliers can mislead the learning process of the learner, leading to sub-optimal performance. In this paper, we propose a novel Graduated Escalation (GE) loss that uses a truncation strategy to handle outliers. The GE loss can help the model to detect and eliminate the outliers in the training process. Adhering to this concept, we present a new robust support vector ordinal regression (RSVOR) model that is robust to outliers. Existing robust support vector ordinal regression models, like robust classification models, often use a truncation function to ensure the model's robustness to outliers. However, this simple truncation function ignores the unique order relationship in ordinal regression. The Graduated Escalation function of RSVOR ensures robustness to outliers while also preserving the order information, a characteristic often overlooked by existing robust ordinal regression models. A binary weight matrix is used in RSVOR to identify and eliminate outliers to improve the robustness against outliers. Additionally, we develop a new optimization algorithm based on difference of convex (DC) algorithm to efficiently minimize the GE loss objective. Theoretical results demonstrate the convergence of our optimization algorithm. Extensive empirical results show that our method outperforms state-of-the-art ordinal regression methods on datasets containing outliers.
KW - Difference of convex algorithm
KW - Graduated escalation loss
KW - Ordinal regression
KW - Robust to outlier
KW - Support vector ordinal regression
UR - http://www.scopus.com/inward/record.url?scp=105005077215&partnerID=8YFLogxK
U2 - 10.1016/j.ins.2025.122277
DO - 10.1016/j.ins.2025.122277
M3 - 文章
AN - SCOPUS:105005077215
SN - 0020-0255
VL - 717
JO - Information Sciences
JF - Information Sciences
M1 - 122277
ER -