Audio-Visual Kinship Verification in the Wild

Xiaoting Wu, Eric Granger, Tomi H. Kinnunen, Xiaoyi Feng, Abdenour Hadid

科研成果: 书/报告/会议事项章节会议稿件同行评审

14 引用 (Scopus)

摘要

Kinship verification is a challenging problem, where recognition systems are trained to establish a kin relation between two individuals based on facial images or videos. However, due to variations in capture conditions (background, pose, expression, illumination and occlusion), state-of-the-art systems currently provide a low level of accuracy. As in many visual recognition and affective computing applications, kinship verification may benefit from a combination of discriminant information extracted from both video and audio signals. In this paper, we investigate for the first time the fusion audio-visual information from both face and voice modalities to improve kinship verification accuracy. First, we propose a new multi-modal kinship dataset called TALking KINship (TALKIN), that is comprised of several pairs of video sequences with subjects talking. State-of-the-art conventional and deep learning models are assessed and compared for kinship verification using this dataset. Finally, we propose a deep Siamese network for multi-modal fusion of kinship relations. Experiments with the TALKIN dataset indicate that the proposed Siamese network provides a significantly higher level of accuracy over baseline uni-modal and multi-modal fusion techniques for kinship verification. Results also indicate that audio (vocal) information is complementary and useful for kinship verification problem.

源语言英语
主期刊名2019 International Conference on Biometrics, ICB 2019
出版商Institute of Electrical and Electronics Engineers Inc.
ISBN(电子版)9781728136400
DOI
出版状态已出版 - 6月 2019
活动2019 International Conference on Biometrics, ICB 2019 - Crete, 希腊
期限: 4 6月 20197 6月 2019

出版系列

姓名2019 International Conference on Biometrics, ICB 2019

会议

会议2019 International Conference on Biometrics, ICB 2019
国家/地区希腊
Crete
时期4/06/197/06/19

指纹

探究 'Audio-Visual Kinship Verification in the Wild' 的科研主题。它们共同构成独一无二的指纹。

引用此