Experimental Results on Multi-modal Deepfake Detection

Sara Concas; Jie Gao; Carlo Cuccu; Giulia Orrù; Xiaoyi Feng; Gian Luca Marcialis; Giovanni Puglisi; Fabio Roli

doi:10.1007/978-3-031-06430-2_14

Experimental Results on Multi-modal Deepfake Detection

Sara Concas, Jie Gao, Carlo Cuccu, Giulia Orrù, Xiaoyi Feng, Gian Luca Marcialis, Giovanni Puglisi, Fabio Roli

School of Electronics and Information

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Scopus citations

Abstract

The advantages of deepfakes in many applications are counterbalanced by their malicious use, for example, in reply-attacks against a biometric system, identification evasion, and people harassment, when they are widespread in social networks and chatting platforms (cyberbullying) as recently documented in newspapers. Due to its “arms-race” nature, deepfake detection systems are often trained on a certain class of deepfakes and showed their limits on never-seen-before classes. In order to shed some light on this problem, we explore the benefits of a multi-modal deepfake detection system. We adopted simple fusion rules, which showed their effectiveness in many applications, for example, biometric recognition, to exploit the complementary of different individual classifiers, and derive some possible guidelines for the designer.

Original language	English
Title of host publication	Image Analysis and Processing – ICIAP 2022 - 21st International Conference, 2022, Proceedings
Editors	Stan Sclaroff, Cosimo Distante, Marco Leo, Giovanni M. Farinella, Federico Tombari
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	164-175
Number of pages	12
ISBN (Print)	9783031064296
DOIs	https://doi.org/10.1007/978-3-031-06430-2_14
State	Published - 2022
Event	21st International Conference on Image Analysis and Processing, ICIAP 2022 - Lecce, Italy Duration: 23 May 2022 → 27 May 2022

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	13232 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	21st International Conference on Image Analysis and Processing, ICIAP 2022
Country/Territory	Italy
City	Lecce
Period	23/05/22 → 27/05/22

Keywords

Deepfake
Multiple classifiers
Pattern recognition

Access to Document

10.1007/978-3-031-06430-2_14

Cite this

Concas, S., Gao, J., Cuccu, C., Orrù, G., Feng, X., Marcialis, G. L., Puglisi, G., & Roli, F. (2022). Experimental Results on Multi-modal Deepfake Detection. In S. Sclaroff, C. Distante, M. Leo, G. M. Farinella, & F. Tombari (Eds.), Image Analysis and Processing – ICIAP 2022 - 21st International Conference, 2022, Proceedings (pp. 164-175). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13232 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-06430-2_14

Concas, Sara ; Gao, Jie ; Cuccu, Carlo et al. / Experimental Results on Multi-modal Deepfake Detection. Image Analysis and Processing – ICIAP 2022 - 21st International Conference, 2022, Proceedings. editor / Stan Sclaroff ; Cosimo Distante ; Marco Leo ; Giovanni M. Farinella ; Federico Tombari. Springer Science and Business Media Deutschland GmbH, 2022. pp. 164-175 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{6e5cd0523fa5467ea760f822958bcb84,

title = "Experimental Results on Multi-modal Deepfake Detection",

abstract = "The advantages of deepfakes in many applications are counterbalanced by their malicious use, for example, in reply-attacks against a biometric system, identification evasion, and people harassment, when they are widespread in social networks and chatting platforms (cyberbullying) as recently documented in newspapers. Due to its “arms-race” nature, deepfake detection systems are often trained on a certain class of deepfakes and showed their limits on never-seen-before classes. In order to shed some light on this problem, we explore the benefits of a multi-modal deepfake detection system. We adopted simple fusion rules, which showed their effectiveness in many applications, for example, biometric recognition, to exploit the complementary of different individual classifiers, and derive some possible guidelines for the designer.",

keywords = "Deepfake, Multiple classifiers, Pattern recognition",

author = "Sara Concas and Jie Gao and Carlo Cuccu and Giulia Orr{\`u} and Xiaoyi Feng and Marcialis, {Gian Luca} and Giovanni Puglisi and Fabio Roli",

note = "Publisher Copyright: {\textcopyright} 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.; 21st International Conference on Image Analysis and Processing, ICIAP 2022 ; Conference date: 23-05-2022 Through 27-05-2022",

year = "2022",

doi = "10.1007/978-3-031-06430-2_14",

language = "英语",

isbn = "9783031064296",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "164--175",

editor = "Stan Sclaroff and Cosimo Distante and Marco Leo and Farinella, {Giovanni M.} and Federico Tombari",

booktitle = "Image Analysis and Processing – ICIAP 2022 - 21st International Conference, 2022, Proceedings",

}

Concas, S, Gao, J, Cuccu, C, Orrù, G, Feng, X, Marcialis, GL, Puglisi, G & Roli, F 2022, Experimental Results on Multi-modal Deepfake Detection. in S Sclaroff, C Distante, M Leo, GM Farinella & F Tombari (eds), Image Analysis and Processing – ICIAP 2022 - 21st International Conference, 2022, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 13232 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 164-175, 21st International Conference on Image Analysis and Processing, ICIAP 2022, Lecce, Italy, 23/05/22. https://doi.org/10.1007/978-3-031-06430-2_14

Experimental Results on Multi-modal Deepfake Detection. / Concas, Sara; Gao, Jie; Cuccu, Carlo et al.
Image Analysis and Processing – ICIAP 2022 - 21st International Conference, 2022, Proceedings. ed. / Stan Sclaroff; Cosimo Distante; Marco Leo; Giovanni M. Farinella; Federico Tombari. Springer Science and Business Media Deutschland GmbH, 2022. p. 164-175 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13232 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Experimental Results on Multi-modal Deepfake Detection

AU - Concas, Sara

AU - Gao, Jie

AU - Cuccu, Carlo

AU - Orrù, Giulia

AU - Feng, Xiaoyi

AU - Marcialis, Gian Luca

AU - Puglisi, Giovanni

AU - Roli, Fabio

PY - 2022

Y1 - 2022

N2 - The advantages of deepfakes in many applications are counterbalanced by their malicious use, for example, in reply-attacks against a biometric system, identification evasion, and people harassment, when they are widespread in social networks and chatting platforms (cyberbullying) as recently documented in newspapers. Due to its “arms-race” nature, deepfake detection systems are often trained on a certain class of deepfakes and showed their limits on never-seen-before classes. In order to shed some light on this problem, we explore the benefits of a multi-modal deepfake detection system. We adopted simple fusion rules, which showed their effectiveness in many applications, for example, biometric recognition, to exploit the complementary of different individual classifiers, and derive some possible guidelines for the designer.

AB - The advantages of deepfakes in many applications are counterbalanced by their malicious use, for example, in reply-attacks against a biometric system, identification evasion, and people harassment, when they are widespread in social networks and chatting platforms (cyberbullying) as recently documented in newspapers. Due to its “arms-race” nature, deepfake detection systems are often trained on a certain class of deepfakes and showed their limits on never-seen-before classes. In order to shed some light on this problem, we explore the benefits of a multi-modal deepfake detection system. We adopted simple fusion rules, which showed their effectiveness in many applications, for example, biometric recognition, to exploit the complementary of different individual classifiers, and derive some possible guidelines for the designer.

KW - Deepfake

KW - Multiple classifiers

KW - Pattern recognition

UR - http://www.scopus.com/inward/record.url?scp=85130941210&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-06430-2_14

DO - 10.1007/978-3-031-06430-2_14

M3 - 会议稿件

AN - SCOPUS:85130941210

SN - 9783031064296

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 164

EP - 175

BT - Image Analysis and Processing – ICIAP 2022 - 21st International Conference, 2022, Proceedings

A2 - Sclaroff, Stan

A2 - Distante, Cosimo

A2 - Leo, Marco

A2 - Farinella, Giovanni M.

A2 - Tombari, Federico

PB - Springer Science and Business Media Deutschland GmbH

T2 - 21st International Conference on Image Analysis and Processing, ICIAP 2022

Y2 - 23 May 2022 through 27 May 2022

ER -

Concas S, Gao J, Cuccu C, Orrù G, Feng X, Marcialis GL et al. Experimental Results on Multi-modal Deepfake Detection. In Sclaroff S, Distante C, Leo M, Farinella GM, Tombari F, editors, Image Analysis and Processing – ICIAP 2022 - 21st International Conference, 2022, Proceedings. Springer Science and Business Media Deutschland GmbH. 2022. p. 164-175. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-06430-2_14