Deep bidirectional LSTM modeling of timbre and prosody for emotional voice conversion

Emotional voice conversion aims at converting speech from one emotion state to another. This paper proposes to model tim- bre and prosody features using a deep bidirectional long short- term memory (DBLSTM) for emotional voice conversion. A continuous wavelet transform (CWT) representation of funda- mental frequency (F0) and energy contour are used for prosody modeling. Specifically, we use CWT to decompose F0 into a five-scale representation, and decompose energy contour into a ten-scale representation, where each feature scale corresponds to a temporal scale. Both spectrum and prosody (F0 and energy contour) features are simultaneously converted by a sequence to sequence conversion method with DBLSTM model, which captures both frame-wise and long-range relationship between source and target voice. The converted speech signals are e- valuated both objectively and subjectively, which confirms the effectiveness of the proposed method.