EMS-Net: Ensemble of Multiscale Convolutional Neural Networks for Classification of Breast Cancer Histology Images

Histology images analysis resulted from needle biopsy serves as the gold standard for breast cancer diagnosis. Deep learning-based classification of breast tissues in histology images, however, is less accurate, due to the lack of adequate training data and ignoring the multiscale structural and textural information. In this paper, we propose the Ensemble of MultiScale convolutional neural Networks (EMS-Net) to classify hematoxylin–eosin stained breast histopathological microscopy images into four categories, including normal tissue, benign lesion, in situ carcinoma, invasive carcinoma. We first convert each image to multiple scales, and then use the training patches cropped and augmented at each scale to fine-tune the pre-trained DenseNet-161, ResNet-152, and ResNet-101, respectively. We find that a combination of three fine-tuned models is more accurate than other combinations, and use them to form an ensemble model. We evaluated our algorithm against three recent methods on the BACH challenge dataset. It shows that the proposed EMS-Net algorithm achieved an accuracy of 91.75 ± 2.32% in the five-fold cross validation using 400 training images, which is higher than the accuracy of other three algorithms, and also achieved an accuracy of 90.00% in the online verification using 100 testing images.