A kullback-leibler divergence based recurrent mixture density network for acoustic modeling in emotional statistical parametric speech synthesis

Xiaochun An, Yuchao Zhang, Bing Liu, Liumeng Xue, Lei Xie

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations

Abstract

This paper proposes a Kullback-Leibler divergence (KLD) based recurrent mixture density network (RMDN) approach for acoustic modeling in emotional statistical parametric speech synthesis (SPSS), which aims at improving model accuracy and emotion naturalness. First, to improve model accuracy, we propose to use RMDN as acoustic model, which combines an LSTM with a mixture density network (MDN). Adding mixture density layer allows us to do multimodal regression as well as to predict variances, thus modeling more accurate probability density functions of acoustic features. Second, we further introduce Kullback-Leibler divergence regularization in model training. Inspired by KLD’s success in acoustic model adaptation, we aim to improve the emotion naturalness by maximizing the distances between the distributions of emotional speech and neutral speech. Objective and subjective evaluations show that the proposed approach improves the prediction accuracy of acoustic features and the naturalness of the synthesized emotional speech.

Original languageEnglish
Title of host publicationASMMC-MMAC 2018 - Proceedings of the Joint Workshop of the 4th Workshop on Affective Social Multimedia Computing and 1st Multi-Modal Affective Computing of Large-Scale Multimedia Data, Co-located with MM 2018
PublisherAssociation for Computing Machinery, Inc
Pages1-6
Number of pages6
ISBN (Electronic)9781450359856
DOIs
StatePublished - 19 Oct 2018
EventJoint Workshop of the 4th Workshop on Affective Social Multimedia Computing and 1st Multi-Modal Affective Computing of Large-Scale Multimedia Data Workshop, ASMMC-MMAC 2018 - Seoul, Korea, Republic of
Duration: 26 Oct 2018 → …

Publication series

NameASMMC-MMAC 2018 - Proceedings of the Joint Workshop of the 4th Workshop on Affective Social Multimedia Computing and 1st Multi-Modal Affective Computing of Large-Scale Multimedia Data, Co-located with MM 2018

Conference

ConferenceJoint Workshop of the 4th Workshop on Affective Social Multimedia Computing and 1st Multi-Modal Affective Computing of Large-Scale Multimedia Data Workshop, ASMMC-MMAC 2018
Country/TerritoryKorea, Republic of
CitySeoul
Period26/10/18 → …

Keywords

  • Emotional statistical parametric speech synthesis
  • KLD-RMDN
  • LSTM
  • Recurrent mixture density network

Fingerprint

Dive into the research topics of 'A kullback-leibler divergence based recurrent mixture density network for acoustic modeling in emotional statistical parametric speech synthesis'. Together they form a unique fingerprint.

Cite this