Group-wise attentive enhancements for unsupervised feature selection

Unsupervised feature selection (UFS) methods have been successful in dealing with high-dimensional data. However, they heavily depend on the assumption that the importance of different features is indiscriminate, which is not applicable in reality. In addition, the existing UFS methods usually employ ℓ_2,1-norm regularization terms to make the model sparse and evaluate the scores of features, which are computationally complex and tend to make the model fall into a local optimum. To address the above drawbacks, we propose a novel Group-wise Attentive Enhancements (GAE) for Unsupervised Feature Selection method, which could learn the similarity graph, the feature attention matrix, and the row-sparse orthogonal projection matrix simultaneously. Firstly, different features play unequal roles in the data, so the feature attention module designed can adaptively exert attention, i.e., weights, to different features and learn discriminative feature subsets. In addition, to minimize the computational difficulty and the possibility of the model falling into sub-optimal solutions, we impose a ℓ_2,0-norm sparse constraint on the orthogonal projection matrix. Finally, the maximum information entropy concept is introduced to avoid the emergence of trivial solutions, which enables the learned similarity graph to delicately depict the data's local structure. To solve the GAE model which is an NP-hard problem, we propose an efficient iterative optimization method to obtain the global optimal solution of the projection matrix. In addition, we prove the convergence and computational complexity, and conduct extensive experiments to verify the superiority of the proposed method over advanced feature selection methods.