Lip contour extraction method based on multiple active shape model for audio visual speech recognition

Lei Xie, Wei Feng, Rongchun Zhao

科研成果: 期刊稿件文章同行评审

1 引用 (Scopus)

摘要

In audio visual speech recognition and lipreading, the widely used ASM (active shape model) for lip contour extraction suffers from the lack of robustness and cannot extract the exact lip contours due to the various mouth shape changes when uttering. We present a more robust model-multiple active shape model (MASM). The model classifies the mouth shapes into closed mouth set, half-opened mouth set, and round mouth set. An independent ASM is built for each different set with a tiny set of the training data. The MASM contour extraction algorithm automatically selects the best accurate lip contour from multiple shape searching procedures. Considering the consecutive changes of the mouth, a method for smoothing lip contours is also presented to correct the contour extraction errors. Experimental results from AVCONDIG database show that extraction accuracy achieved by the MASM is higher than that of conventional ASM 13%. The combination of the MASM and the contour-smoothing method leads to another 7% accuracy improvement. With the fusion of the exact lip contour feature and audio MFCC (mel frequency cepstral coefficients) feature, the average word recognition rates of the considered connected-digits speech recognition task are considerably increased under noisy acoustic conditions.

源语言英语
页(从-至)674-678
页数5
期刊Xibei Gongye Daxue Xuebao/Journal of Northwestern Polytechnical University
22
5
出版状态已出版 - 10月 2004

指纹

探究 'Lip contour extraction method based on multiple active shape model for audio visual speech recognition' 的科研主题。它们共同构成独一无二的指纹。

引用此