Hybrid genetic and variational expectation-maximization algorithm for Gaussian-mixture-model-based brain MR image segmentation

The expectation-maximization (EM) algorithm has been widely applied to the estimation of Gaussian mixture model (GMM) in brain MR image segmentation. However, the EM algorithm is deterministic and intrinsically prone to overfitting the training data and being trapped in local optima. In this paper, we propose a hybrid genetic and variational EM (GA-VEM) algorithm for brain MR image segmentation. In this approach, the VEM algorithm is performed to estimate the GMM, and the GA is employed to initialize the hyperparameters of the conjugate prior distributions of GMM parameters involved in the VEM algorithm. Since GA has the potential to achieve global optimization and VEM can steadily avoid overfitting, the hybrid GA-VEM algorithm is capable of overcoming the drawbacks of traditional EM-based methods. We compared our approach to the EM-based, VEM-based, and GA-EM based segmentation algorithms, and the segmentation routines used in the statistical parametric mapping package and FMRIB Software Library in 20 low-resolution and 17 high-resolution brain MR studies. Our results show that the proposed approach can improve substantially the performance of brain MR image segmentation.