Robust support vector ordinal regression

Ordinal regression is a unique supervised problem where the labels exhibit a natural order, setting it apart from multi-class classification and metric regression. While Support Vector Ordinal Regression with Explicit Constraints (SVOREX) is a widely used model for ordinal regression, it is highly sensitive to outliers in the training set. In many pattern recognition and machine learning tasks, outliers are often present in the training set. These outliers can mislead the learning process of the learner, leading to sub-optimal performance. In this paper, we propose a novel Graduated Escalation (GE) loss that uses a truncation strategy to handle outliers. The GE loss can help the model to detect and eliminate the outliers in the training process. Adhering to this concept, we present a new robust support vector ordinal regression (RSVOR) model that is robust to outliers. Existing robust support vector ordinal regression models, like robust classification models, often use a truncation function to ensure the model's robustness to outliers. However, this simple truncation function ignores the unique order relationship in ordinal regression. The Graduated Escalation function of RSVOR ensures robustness to outliers while also preserving the order information, a characteristic often overlooked by existing robust ordinal regression models. A binary weight matrix is used in RSVOR to identify and eliminate outliers to improve the robustness against outliers. Additionally, we develop a new optimization algorithm based on difference of convex (DC) algorithm to efficiently minimize the GE loss objective. Theoretical results demonstrate the convergence of our optimization algorithm. Extensive empirical results show that our method outperforms state-of-the-art ordinal regression methods on datasets containing outliers.