Visualize speech: A continuous speech recognition system for facial animation using acoustic visemes

  • Lei Xie
  • , Dongmei Jiang
  • , Ravyse Ilse
  • , Rongchun Zhao
  • , Verhelst Werner
  • , Sahli Hichem
  • , Conlenis Jan

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

This paper presents an acoustic viseme based continuous speech recognition system for speech driven talking face animation. The system is developed using viseme HMMs with acoustic speech as input only. Triseme HMMs are adopted to reflect the mouth shape contexts. Visual decision trees are introduced to get robust parameter training for triseme HMMs with the limited training data. In the tree building process, methods based on lip rounding and similarity of viseme shapes are introduced to design visual questions. The results from objective and subjective evaluations show that the talking face animation based on the speech recognition system provided by this paper outperforms the conventional phoneme based one, and it is possible to obtain visually relevant speech segmentation information from acoustic speech signal only.

Original languageEnglish
Title of host publicationProceedings of 2003 International Conference on Neural Networks and Signal Processing, ICNNSP'03
Pages872-875
Number of pages4
DOIs
StatePublished - 2003
Event2003 International Conference on Neural Networks and Signal Processing, ICNNSP'03 - Nanjing, China
Duration: 14 Dec 200317 Dec 2003

Publication series

NameProceedings of 2003 International Conference on Neural Networks and Signal Processing, ICNNSP'03
Volume2

Conference

Conference2003 International Conference on Neural Networks and Signal Processing, ICNNSP'03
Country/TerritoryChina
CityNanjing
Period14/12/0317/12/03

Fingerprint

Dive into the research topics of 'Visualize speech: A continuous speech recognition system for facial animation using acoustic visemes'. Together they form a unique fingerprint.

Cite this