跳到主要导航 跳到搜索 跳到主要内容

Adversarial Feature Learning and Unsupervised Clustering Based Speech Synthesis for Found Data with Acoustic and Textual Noise

  • Northwestern Polytechnical University Xian
  • ByteDance Ltd.

科研成果: 期刊稿件 › 文章 › 同行评审

10 引用 (Scopus)

摘要

Attention-based sequence-to-sequence (seq2seq) speech synthesis has achieved extraordinary performance. But a studio-quality corpus with manual transcription is necessary to train such seq2seq systems. In this letter, we propose an approach to build high-quality and stable seq2seq based speech synthesis system using challenging found data, where training speech contains noisy interferences (acoustic noise) and texts are imperfect speech recognition transcripts (textual noise). To deal with text-side noise, we propose a VQVAE based heuristic method to compensate erroneous linguistic feature with phonetic information learned directly from speech. As for the speech-side noise, we propose to learn a noise-independent feature in the auto-regressive decoder through adversarial training and data augmentation, which does not need an extra speech enhancement model. Experiments show the effectiveness of the proposed approach in dealing with text-side and speech-side noise. Surpassing the denoising approach based on a state-of-the-art speech enhancement model, our system built on noisy found data can synthesize clean and high-quality speech with MOS close to the system built on the clean counterpart.

源语言英语
期刊论文编号9201416
页(从-至)1730-1734
页数5
期刊IEEE Signal Processing Letters
卷27
DOI
出版状态已出版 - 2020

学术指纹

探究 'Adversarial Feature Learning and Unsupervised Clustering Based Speech Synthesis for Found Data with Acoustic and Textual Noise' 的科研主题。它们共同构成独一无二的学术指纹。

引用此