Jointly using deep model learned features and traditional visual features in a stacked SVM for medical subfigure classification

Classification of diagnose images and illustrations in the literature is a major challenge towards automated literature review and retrieval. Although being widely recognized as the most successful image classification technique, deep learning models, however, may need to be complemented by traditional visual features to solve this problem, in which there are intra-class variation, inter-class similarity and a small training dataset. In this paper, we propose an approach to classifying diagnose images and biomedical publication illustrations. This algorithm jointly uses the image representations learned by three pre-trained deep convolutional neural network models and ten types of traditional visual features in a stacked support vector machine (SVM) classifier. We have evaluated this algorithm on the ImageCLEF 2016 Subfigure Classification dataset and achieved an accuracy of 85.62%, which is higher than the top performance of purely visual approaches in this challenge.