Triseme decision trees in the continuous speech recognition system for a talking head

Dong Mei Jiang; Lei Xie; Ilse Ravyse; Rong Chun Zhao; Hichem Sahli; Jan Cornelis

Triseme decision trees in the continuous speech recognition system for a talking head

Dong Mei Jiang
, Lei Xie
, Ilse Ravyse
, Rong Chun Zhao
, Hichem Sahli
, Jan Cornelis

School of Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

3 Scopus citations

Abstract

In this paper, we present a viseme (the basic speech units in the visual domain) based continuous speech recognition system, which segments speech into viseme sequences with timing boundaries to drive a talking head. In the viseme Hidden Markov Model (HMM) training, the instances of a viseme with different contexts are formulated as trisemes. Based on the mouth shape parameters Liprounding and the defined viseme similarity weight (VSW) from the 3D viseme facial models, 166 questions concerning the viseme contexts are designed to build triseme decision trees to tie the states of the trisemes with similar contexts, so that they can share the same parameters. To evaluate the system performance, the image related measurements are also taken to evaluate the resulting viseme sequences, with 'jerky instances' in Liprounding and VSW graphs evaluating their smoothness. Results show that compared to the phoneme based system, the tied-state triseme based speech recognition system gives talking head animation with smoother and more plausible mouth shapes.

Original language	English
Title of host publication	Proceedings of 2002 International Conference on Machine Learning and Cybernetics
Pages	2097-2101
Number of pages	5
State	Published - 2002
Event	Proceedings of 2002 International Conference on Machine Learning and Cybernetics - Beijing, China Duration: 4 Nov 2002 → 5 Nov 2002

Publication series

Name	Proceedings of 2002 International Conference on Machine Learning and Cybernetics
Volume	4

Conference

Conference	Proceedings of 2002 International Conference on Machine Learning and Cybernetics
Country/Territory	China
City	Beijing
Period	4/11/02 → 5/11/02

Keywords

Jerky instances
Liprounding
Triseme decision tree
Viseme
Viseme similarity weight

Cite this

@inproceedings{59acff0c039042588b5e3f02ac6218f7,

title = "Triseme decision trees in the continuous speech recognition system for a talking head",

abstract = "In this paper, we present a viseme (the basic speech units in the visual domain) based continuous speech recognition system, which segments speech into viseme sequences with timing boundaries to drive a talking head. In the viseme Hidden Markov Model (HMM) training, the instances of a viseme with different contexts are formulated as trisemes. Based on the mouth shape parameters Liprounding and the defined viseme similarity weight (VSW) from the 3D viseme facial models, 166 questions concerning the viseme contexts are designed to build triseme decision trees to tie the states of the trisemes with similar contexts, so that they can share the same parameters. To evaluate the system performance, the image related measurements are also taken to evaluate the resulting viseme sequences, with 'jerky instances' in Liprounding and VSW graphs evaluating their smoothness. Results show that compared to the phoneme based system, the tied-state triseme based speech recognition system gives talking head animation with smoother and more plausible mouth shapes.",

keywords = "Jerky instances, Liprounding, Triseme decision tree, Viseme, Viseme similarity weight",

author = "Jiang, \{Dong Mei\} and Lei Xie and Ilse Ravyse and Zhao, \{Rong Chun\} and Hichem Sahli and Jan Cornelis",

year = "2002",

language = "英语",

isbn = "0780375084",

series = "Proceedings of 2002 International Conference on Machine Learning and Cybernetics",

pages = "2097--2101",

booktitle = "Proceedings of 2002 International Conference on Machine Learning and Cybernetics",

note = "Proceedings of 2002 International Conference on Machine Learning and Cybernetics ; Conference date: 04-11-2002 Through 05-11-2002",

}

Jiang, DM, Xie, L, Ravyse, I, Zhao, RC, Sahli, H & Cornelis, J 2002, Triseme decision trees in the continuous speech recognition system for a talking head. in Proceedings of 2002 International Conference on Machine Learning and Cybernetics. Proceedings of 2002 International Conference on Machine Learning and Cybernetics, vol. 4, pp. 2097-2101, Proceedings of 2002 International Conference on Machine Learning and Cybernetics, Beijing, China, 4/11/02.

Triseme decision trees in the continuous speech recognition system for a talking head. / Jiang, Dong Mei; Xie, Lei; Ravyse, Ilse et al.
Proceedings of 2002 International Conference on Machine Learning and Cybernetics. 2002. p. 2097-2101 (Proceedings of 2002 International Conference on Machine Learning and Cybernetics; Vol. 4).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Triseme decision trees in the continuous speech recognition system for a talking head

AU - Jiang, Dong Mei

AU - Xie, Lei

AU - Ravyse, Ilse

AU - Zhao, Rong Chun

AU - Sahli, Hichem

AU - Cornelis, Jan

PY - 2002

Y1 - 2002

N2 - In this paper, we present a viseme (the basic speech units in the visual domain) based continuous speech recognition system, which segments speech into viseme sequences with timing boundaries to drive a talking head. In the viseme Hidden Markov Model (HMM) training, the instances of a viseme with different contexts are formulated as trisemes. Based on the mouth shape parameters Liprounding and the defined viseme similarity weight (VSW) from the 3D viseme facial models, 166 questions concerning the viseme contexts are designed to build triseme decision trees to tie the states of the trisemes with similar contexts, so that they can share the same parameters. To evaluate the system performance, the image related measurements are also taken to evaluate the resulting viseme sequences, with 'jerky instances' in Liprounding and VSW graphs evaluating their smoothness. Results show that compared to the phoneme based system, the tied-state triseme based speech recognition system gives talking head animation with smoother and more plausible mouth shapes.

AB - In this paper, we present a viseme (the basic speech units in the visual domain) based continuous speech recognition system, which segments speech into viseme sequences with timing boundaries to drive a talking head. In the viseme Hidden Markov Model (HMM) training, the instances of a viseme with different contexts are formulated as trisemes. Based on the mouth shape parameters Liprounding and the defined viseme similarity weight (VSW) from the 3D viseme facial models, 166 questions concerning the viseme contexts are designed to build triseme decision trees to tie the states of the trisemes with similar contexts, so that they can share the same parameters. To evaluate the system performance, the image related measurements are also taken to evaluate the resulting viseme sequences, with 'jerky instances' in Liprounding and VSW graphs evaluating their smoothness. Results show that compared to the phoneme based system, the tied-state triseme based speech recognition system gives talking head animation with smoother and more plausible mouth shapes.

KW - Jerky instances

KW - Liprounding

KW - Triseme decision tree

KW - Viseme

KW - Viseme similarity weight

UR - https://www.scopus.com/pages/publications/0036921857

M3 - 会议稿件

AN - SCOPUS:0036921857

SN - 0780375084

T3 - Proceedings of 2002 International Conference on Machine Learning and Cybernetics

SP - 2097

EP - 2101

BT - Proceedings of 2002 International Conference on Machine Learning and Cybernetics

T2 - Proceedings of 2002 International Conference on Machine Learning and Cybernetics

Y2 - 4 November 2002 through 5 November 2002

ER -

Triseme decision trees in the continuous speech recognition system for a talking head

Abstract

Publication series

Conference

Keywords

Other files and links

Fingerprint

Cite this