TY - GEN
T1 - Glow-WaveGAN
T2 - 22nd Annual Conference of the International Speech Communication Association, INTERSPEECH 2021
AU - Cong, Jian
AU - Yang, Shan
AU - Xie, Lei
AU - Su, Dan
N1 - Publisher Copyright:
© 2021 ISCA
PY - 2021
Y1 - 2021
N2 - Current two-stage TTS framework typically integrates an acoustic model with a vocoder - the acoustic model predicts a low resolution intermediate representation such as Mel-spectrum while the vocoder generates waveform from the intermediate representation. Although the intermediate representation is served as a bridge, there still exists critical mismatch between the acoustic model and the vocoder as they are commonly separately learned and work on different distributions of representation, leading to inevitable artifacts in the synthesized speech. In this work, different from using pre-designed intermediate representation in most previous studies, we propose to use VAE combining with GAN to learn a latent representation directly from speech and then utilize a flow-based acoustic model to model the distribution of the latent representation from text. In this way, the mismatch problem is migrated as the two stages work on the same distribution. Results demonstrate that the flow-based acoustic model can exactly model the distribution of our learned speech representation and the proposed TTS framework, namely Glow-WaveGAN, can produce high fidelity speech outperforming the state-of-the-art GAN-based model.
AB - Current two-stage TTS framework typically integrates an acoustic model with a vocoder - the acoustic model predicts a low resolution intermediate representation such as Mel-spectrum while the vocoder generates waveform from the intermediate representation. Although the intermediate representation is served as a bridge, there still exists critical mismatch between the acoustic model and the vocoder as they are commonly separately learned and work on different distributions of representation, leading to inevitable artifacts in the synthesized speech. In this work, different from using pre-designed intermediate representation in most previous studies, we propose to use VAE combining with GAN to learn a latent representation directly from speech and then utilize a flow-based acoustic model to model the distribution of the latent representation from text. In this way, the mismatch problem is migrated as the two stages work on the same distribution. Results demonstrate that the flow-based acoustic model can exactly model the distribution of our learned speech representation and the proposed TTS framework, namely Glow-WaveGAN, can produce high fidelity speech outperforming the state-of-the-art GAN-based model.
KW - Generative adversarial network
KW - Speech representations
KW - Speech synthesis
KW - Variational auto-encoder
UR - http://www.scopus.com/inward/record.url?scp=85118309081&partnerID=8YFLogxK
U2 - 10.21437/Interspeech.2021-414
DO - 10.21437/Interspeech.2021-414
M3 - 会议稿件
AN - SCOPUS:85118309081
T3 - Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH
SP - 3241
EP - 3245
BT - 22nd Annual Conference of the International Speech Communication Association, INTERSPEECH 2021
PB - International Speech Communication Association
Y2 - 30 August 2021 through 3 September 2021
ER -