The Conversational Short-phrase Speaker Diarization (CSSD) Task: Dataset, Evaluation Metric and Baselines

Gaofeng Cheng, Yifan Chen, Runyan Yang, Qingxuan Li, Zehui Yang, Lingxuan Ye, Pengyuan Zhang, Qingqing Zhang, Lei Xie, Yanmin Qian, Kong Aik Lee, Yonghong Yan

科研成果: 书/报告/会议事项章节会议稿件同行评审

5 引用 (Scopus)

摘要

The conversation scenario is one of the most important and most challenging scenarios for speech processing technologies because people in conversation respond to each other in a casual style. Detecting the speech activities of each person in a conversation is vital to downstream tasks, like natural language processing, machine translation, etc. People refer to the detection technology of 'who speak when' as speaker diarization (SD). Traditionally, diarization error rate (DER) has been used as the standard evaluation metric of SD systems for a long time. However, DER fails to give enough importance to short conversational phrases, which are short but important on the semantic level. Also, a carefully and accurately manually-annotated testing dataset suitable for evaluating the conversational SD technologies is still unavailable in the speech community. In this paper, we design and describe the Conversational Short-phrases Speaker Diarization (CSSD) task, which consists of training and testing datasets, evaluation metric and baselines. In the dataset aspect, despite the previously open-sourced 180-hour conversational MagicData-RAMC dataset, we prepare an individual 20-hour conversational speech test dataset with carefully and artificially verified speakers timestamps annotations for the CSSD task. In the metric aspect, we design the new conversational DER (CDER) evaluation metric, which calculates the SD accuracy at the utterance level. In the baseline aspect, we adopt a commonly used method: Variational Bayes HMM x-vector system, as the baseline of the CSSD task. Our evaluation metric is publicly available at https://github.com/SpeechClub/CDER_Metric.

源语言英语
主期刊名2022 13th International Symposium on Chinese Spoken Language Processing, ISCSLP 2022
编辑Kong Aik Lee, Hung-yi Lee, Yanfeng Lu, Minghui Dong
出版商Institute of Electrical and Electronics Engineers Inc.
488-492
页数5
ISBN(电子版)9798350397963
DOI
出版状态已出版 - 2022
活动13th International Symposium on Chinese Spoken Language Processing, ISCSLP 2022 - Singapore, 新加坡
期限: 11 12月 202214 12月 2022

出版系列

姓名2022 13th International Symposium on Chinese Spoken Language Processing, ISCSLP 2022

会议

会议13th International Symposium on Chinese Spoken Language Processing, ISCSLP 2022
国家/地区新加坡
Singapore
时期11/12/2214/12/22

指纹

探究 'The Conversational Short-phrase Speaker Diarization (CSSD) Task: Dataset, Evaluation Metric and Baselines' 的科研主题。它们共同构成独一无二的指纹。

引用此