BPkNN: K-Nearest Neighbor Classifier with Pairwise Distance Metrics and Belief Function Theory

The k -nearest neighbor (kNN) rule is one of the most popular classification algorithms in pattern recognition field because it is very simple to understand but works quite well in practice. However, the performance of the k NN rule depends critically on its being given a good distance metric over the input space, especially in small data set situations. In this paper, a new k NN-based classifier, called BP k NN, is developed based on pairwise distance metrics and belief function theory. The idea of the proposal is that instead of learning a global distance metric, we first decompose it into learning a group of pairwise distance metrics. Then, based on each learned pairwise distance metric, a pairwise k NN ( \text{P}k NN) sub-classifier can be adaptively designed to separate two classes. Finally, a polychotomous classification problem is solved by combining the outputs of these \text{P}k NN sub-classifiers in belief function framework. The BP k NN classifier improves the classification performance thanks to the new distance metrics which provide more flexibility to design the feature weights and the belief function-based combination method which can better address the uncertainty involved in the outputs of the sub-classifiers. Experimental results based on synthetic and real data sets show that the proposed BP k NN can achieve better classification accuracy in comparison with some state-of-the-art methods.