Boosting algorithm framework for ensemble neural networks based on coordinate descent

Traditional Gradient Boosting Decision Trees (GBDTs) predominantly rely on greedy forward stagewise optimization and decision tree base learners, which often restrict the model to suboptimal local solutions and limit feature representation capabilities. In this paper, we propose a novel framework termed Coordinate Descent Gradient Boosting (CDGB), which fundamentally alters this paradigm. First, we replace standard decision trees with sparse single-hidden-layer neural networks, incorporating ℓ_2,1-norm regularization to adaptively learn compact and diverse subnetworks. Second, unlike the traditional greedy approach that fixes previous learners, we introduce a coordinate descent optimization strategy that enables the joint refinement of all base learners and their weights, theoretically guaranteeing monotonic convergence. Extensive experiments on 14 real-world datasets (covering both classification and regression tasks) demonstrate the robustness of CDGB. Statistical analysis confirms that CDGB achieves the best average rank (ranging from 1.00 to 1.14) across 10 evaluation metrics (e.g., Accuracy, MSE, AUC), significantly outperforming classical baselines (e.g., AdaBoost, Random Forest) and demonstrating superior ranking consistency compared to state-of-the-art libraries such as XGBoost and LightGBM.