T-distributed Stochastic Neighbor Network for unsupervised representation learning

Unsupervised representation learning (URL) is still lack of a reasonable operator (e.g. convolution kernel) for exploring meaningful structural information from generic data including vector, image and tabular data. In this paper, we propose a simple end-to-end T-distributed Stochastic Neighbor Network (TsNet) for URL with clustering downstream task. Concretely, our TsNet model has three major components: (1) an adaptive connectivity distribution learning module is presented to construct a pairwise graph for preserving the local structure of generic data; (2) a T-distributed stochastic neighbor embedding based loss function is designed to learn a transformation between embeddings and original data, which improves the discrimination of representations; (3) a nonlinear parametric mapping is learned via our TsNet on an unsupervised generalized manner, which can address the “out-of-sample” issue. By combining these components, our method is able to considerably outperform previous related unsupervised learning approaches on visualization and clustering of generic data. A simple deep neural network equipped on our model respectively achieves 74.90%, 76.56% ACC and NMI, which is 8% relative improvement over previous state-of-the-art on real single-cell RNA-sequencing (scRNA-seq) datasets clustering.