EXPLORATION OF EFFICIENT HYPERPARAMETERS ADAPTION OF SUPPORT VECTOR REGRESSION FOR AERODYNAMIC DESIGN

Support vector regression (SVR), due to good generalization capability that has been validated in machine learning and pattern recognition, was introduced into aerodynamic design to build surrogate models based on the training data with numerical noise in our former work. However, hyperparameters tuning is still a key problem to solve because it not only has critical impact on the prediction accuracy but also brings high computational cost. Therefore, the hyperparameter optimization model and algorithms are investigated in this work. The objective of the hyperparameter optimization model, generalization error (GE), is obtained via the popular cross validation (CV) method, and compared with the leave-one-out bound (LooB) method due to its high efficiency. The hyperparameter design spaces are plotted and it is found that the curves of GE w.r.t the hyperparameters (the insensitive factor, penalty factor and kernel parameter) are commonly characteristic of multi-modal, large “flat” region and non-smoothness. Therefore, the gradient optimization is not recommended because of its local-search attribute. Three popular global optimization algorithms, including the Genetic Algorithm (GA), Bayesian Optimization (BO) and Covariance Matrix Adaptation Evolution Strategy (CMA-ES), are applied to hyperparameters adaption and compared via a set of benchmark problems to evaluate its training efficiency and prediction accuracy for analytical problems of low/high nonlinearity, based on the samples with low/medium/high intensity noise. The results show that, 1) the design space of hyperparameters tunning is characteristic of multi-modal, large “flat” region and non-smoothness; 2) In terms of accuracy, CMA-ES behaves well for almost all the test cases, while BO is better in the low-dimensional (≤10) cases and is still comparable in the higher-dimensional cases when the noise is not too strong but becomes slightly worse when the noise becomes stronger; 2) In the high-dimensional (>10) cases, the BO algorithm has apparent superiority of efficiency; 3) the parallel CV can not only enable higher mode accuracy but also has high efficiency even faster than LooB. Finally, it is applied in modeling based on the computational aerothermal data and the wind-tunnel experimental data respectively, in which the reasonable results are obtained.