Extracting local binary patterns from image key points: Application to automatic facial expression recognition

Facial expression recognition has widely been investigated in the literature. The need of accurate facial alignment has however limited the deployment of facial expression systems in real-world applications. In this paper, a novel feature extraction method is proposed. It is based on extracting local binary patterns (LBP) from image key points. The face region is first segmented into six facial components (left eye, right eye, left eyebrow, right eyebrow, nose, and mouth). Then, local binary patterns are extracted only from the edge points of each facial component. Finally, the local binary pattern features are collected into a histogram and fed to an SVM classifier for facial expression recognition. Compared to the traditional LBP methodology extracting the features from all image pixels, our proposed approach extracts LBP features only from a set of points of face components, yielding in more compact and discriminative representations. Furthermore, our proposed approach does not require face alignment. Extensive experimental analysis on the commonly used JAFFE facial expression benchmark database showed very promising results, outperforming those of the traditional local binary pattern approach.