跳到主要导航 跳到搜索 跳到主要内容

Expressive-VC: Highly Expressive Voice Conversion with Attention Fusion of Bottleneck and Perturbation Features

  • Ziqian Ning
  • , Qicong Xie
  • , Pengcheng Zhu
  • , Zhichao Wang
  • , Liumeng Xue
  • , Jixun Yao
  • , Lei Xie
  • , Mengxiao Bi
  • Northwestern Polytechnical University Xian
  • Netease Games Ai Lab

科研成果: 书/报告/会议事项章节 › 会议稿件 › 同行评审

22 引用 (Scopus)

摘要

Voice conversion for highly expressive speech is challenging. Current approaches struggle with the balance between speaker similarity, intelligibility, and expressiveness. To address this problem, we propose Expressive-VC, a novel end-to-end voice conversion framework that leverages advantages from both the neural bottleneck feature (BNF) approach and the information perturbation approach. Specifically, we use a BNF encoder and a Perturbed-Wav encoder to form a content extractor to learn linguistic and para-linguistic features respectively, where BNFs come from a robust pre-trained ASR model and the perturbed wave becomes speaker-irrelevant after signal perturbation. We further fuse the linguistic and para-linguistic features through an attention mechanism, where speaker-dependent prosody features are used as the attention query, which results from a prosody encoder with target speaker embedding and normalized pitch and energy of source speech as input. Finally, the decoder consumes the integrated features and the speaker-dependent prosody feature to generate the converted speech. Experiments show that Expressive-VC is superior to several popular systems, achieving both high expressiveness captured from the source speech and high speaker similarity with the target speaker; meanwhile intelligibility is well maintained.

源语言英语
主期刊名ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing, Proceedings
出版商Institute of Electrical and Electronics Engineers Inc.
ISBN(电子版)9781728163277
DOI
出版状态已出版 - 2023
活动48th IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2023 - Rhodes Island, 希腊
期限: 4 6月 2023 → 10 6月 2023

丛书

姓名ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
卷2023-June
ISSN(印刷版)1520-6149

会议

会议48th IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2023
国家/地区希腊
市Rhodes Island
时期4/06/23 → 10/06/23

学术指纹

探究 'Expressive-VC: Highly Expressive Voice Conversion with Attention Fusion of Bottleneck and Perturbation Features' 的科研主题。它们共同构成独一无二的学术指纹。

引用此