Visualize speech: A continuous speech recognition system for facial animation using acoustic visemes

Lei Xie; Dongmei Jiang; Ravyse Ilse; Rongchun Zhao; Verhelst Werner; Sahli Hichem; Conlenis Jan

doi:10.1109/ICNNSP.2003.1280738

Visualize speech: A continuous speech recognition system for facial animation using acoustic visemes

Lei Xie
, Dongmei Jiang
, Ravyse Ilse
, Rongchun Zhao
, Verhelst Werner
, Sahli Hichem
, Conlenis Jan

计算机学院

科研成果: 书/报告/会议事项章节 › 会议稿件 › 同行评审

摘要

This paper presents an acoustic viseme based continuous speech recognition system for speech driven talking face animation. The system is developed using viseme HMMs with acoustic speech as input only. Triseme HMMs are adopted to reflect the mouth shape contexts. Visual decision trees are introduced to get robust parameter training for triseme HMMs with the limited training data. In the tree building process, methods based on lip rounding and similarity of viseme shapes are introduced to design visual questions. The results from objective and subjective evaluations show that the talking face animation based on the speech recognition system provided by this paper outperforms the conventional phoneme based one, and it is possible to obtain visually relevant speech segmentation information from acoustic speech signal only.

源语言	英语
主期刊名	Proceedings of 2003 International Conference on Neural Networks and Signal Processing, ICNNSP'03
页	872-875
页数	4
DOI	https://doi.org/10.1109/ICNNSP.2003.1280738
出版状态	已出版 - 2003
活动	2003 International Conference on Neural Networks and Signal Processing, ICNNSP'03 - Nanjing, 中国期限: 14 12月 2003 → 17 12月 2003

出版系列

姓名	Proceedings of 2003 International Conference on Neural Networks and Signal Processing, ICNNSP'03
卷	2

会议

会议	2003 International Conference on Neural Networks and Signal Processing, ICNNSP'03
国家/地区	中国
市	Nanjing
时期	14/12/03 → 17/12/03

访问文件

10.1109/ICNNSP.2003.1280738

其它文件与链接

链接到 Scopus 的出版物

引用此

Xie, L., Jiang, D., Ilse, R., Zhao, R., Werner, V., Hichem, S., & Jan, C. (2003). Visualize speech: A continuous speech recognition system for facial animation using acoustic visemes. 在 Proceedings of 2003 International Conference on Neural Networks and Signal Processing, ICNNSP'03 (页码 872-875). 文章 1280738 (Proceedings of 2003 International Conference on Neural Networks and Signal Processing, ICNNSP'03; 卷 2). https://doi.org/10.1109/ICNNSP.2003.1280738

@inproceedings{73fdaf36e71b4fe9a7a5180d00743198,

title = "Visualize speech: A continuous speech recognition system for facial animation using acoustic visemes",

abstract = "This paper presents an acoustic viseme based continuous speech recognition system for speech driven talking face animation. The system is developed using viseme HMMs with acoustic speech as input only. Triseme HMMs are adopted to reflect the mouth shape contexts. Visual decision trees are introduced to get robust parameter training for triseme HMMs with the limited training data. In the tree building process, methods based on lip rounding and similarity of viseme shapes are introduced to design visual questions. The results from objective and subjective evaluations show that the talking face animation based on the speech recognition system provided by this paper outperforms the conventional phoneme based one, and it is possible to obtain visually relevant speech segmentation information from acoustic speech signal only.",

author = "Lei Xie and Dongmei Jiang and Ravyse Ilse and Rongchun Zhao and Verhelst Werner and Sahli Hichem and Conlenis Jan",

year = "2003",

doi = "10.1109/ICNNSP.2003.1280738",

language = "英语",

isbn = "0780377028",

series = "Proceedings of 2003 International Conference on Neural Networks and Signal Processing, ICNNSP'03",

pages = "872--875",

booktitle = "Proceedings of 2003 International Conference on Neural Networks and Signal Processing, ICNNSP'03",

note = "2003 International Conference on Neural Networks and Signal Processing, ICNNSP'03 ; Conference date: 14-12-2003 Through 17-12-2003",

}

Xie, L, Jiang, D, Ilse, R, Zhao, R, Werner, V, Hichem, S & Jan, C 2003, Visualize speech: A continuous speech recognition system for facial animation using acoustic visemes. 在 Proceedings of 2003 International Conference on Neural Networks and Signal Processing, ICNNSP'03., 1280738, Proceedings of 2003 International Conference on Neural Networks and Signal Processing, ICNNSP'03, 卷 2, 页码 872-875, 2003 International Conference on Neural Networks and Signal Processing, ICNNSP'03, Nanjing, 中国, 14/12/03. https://doi.org/10.1109/ICNNSP.2003.1280738

Visualize speech: A continuous speech recognition system for facial animation using acoustic visemes. / Xie, Lei; Jiang, Dongmei; Ilse, Ravyse 等.
Proceedings of 2003 International Conference on Neural Networks and Signal Processing, ICNNSP'03. 2003. 页码 872-875 1280738 (Proceedings of 2003 International Conference on Neural Networks and Signal Processing, ICNNSP'03; 卷 2).

科研成果: 书/报告/会议事项章节 › 会议稿件 › 同行评审

TY - GEN

T1 - Visualize speech

T2 - 2003 International Conference on Neural Networks and Signal Processing, ICNNSP'03

AU - Xie, Lei

AU - Jiang, Dongmei

AU - Ilse, Ravyse

AU - Zhao, Rongchun

AU - Werner, Verhelst

AU - Hichem, Sahli

AU - Jan, Conlenis

PY - 2003

Y1 - 2003

N2 - This paper presents an acoustic viseme based continuous speech recognition system for speech driven talking face animation. The system is developed using viseme HMMs with acoustic speech as input only. Triseme HMMs are adopted to reflect the mouth shape contexts. Visual decision trees are introduced to get robust parameter training for triseme HMMs with the limited training data. In the tree building process, methods based on lip rounding and similarity of viseme shapes are introduced to design visual questions. The results from objective and subjective evaluations show that the talking face animation based on the speech recognition system provided by this paper outperforms the conventional phoneme based one, and it is possible to obtain visually relevant speech segmentation information from acoustic speech signal only.

AB - This paper presents an acoustic viseme based continuous speech recognition system for speech driven talking face animation. The system is developed using viseme HMMs with acoustic speech as input only. Triseme HMMs are adopted to reflect the mouth shape contexts. Visual decision trees are introduced to get robust parameter training for triseme HMMs with the limited training data. In the tree building process, methods based on lip rounding and similarity of viseme shapes are introduced to design visual questions. The results from objective and subjective evaluations show that the talking face animation based on the speech recognition system provided by this paper outperforms the conventional phoneme based one, and it is possible to obtain visually relevant speech segmentation information from acoustic speech signal only.

UR - https://www.scopus.com/pages/publications/78650094719

U2 - 10.1109/ICNNSP.2003.1280738

DO - 10.1109/ICNNSP.2003.1280738

M3 - 会议稿件

AN - SCOPUS:78650094719

SN - 0780377028

SN - 9780780377028

T3 - Proceedings of 2003 International Conference on Neural Networks and Signal Processing, ICNNSP'03

SP - 872

EP - 875

BT - Proceedings of 2003 International Conference on Neural Networks and Signal Processing, ICNNSP'03

Y2 - 14 December 2003 through 17 December 2003

ER -

Xie L, Jiang D, Ilse R, Zhao R, Werner V, Hichem S 等. Visualize speech: A continuous speech recognition system for facial animation using acoustic visemes. 在 Proceedings of 2003 International Conference on Neural Networks and Signal Processing, ICNNSP'03. 2003. 页码 872-875. 1280738. (Proceedings of 2003 International Conference on Neural Networks and Signal Processing, ICNNSP'03). doi: 10.1109/ICNNSP.2003.1280738

Visualize speech: A continuous speech recognition system for facial animation using acoustic visemes

摘要

出版系列

会议

访问文件

其它文件与链接

指纹

引用此