TY - JOUR
T1 - Human Cognition-Based Consistency Inference Networks for Multi-Modal Fake News Detection
AU - Wu, Lianwei
AU - Liu, Pusheng
AU - Zhao, Yongqiang
AU - Wang, Peng
AU - Zhang, Yangning
N1 - Publisher Copyright:
© 1989-2012 IEEE.
PY - 2024/1/1
Y1 - 2024/1/1
N2 - The existing models for multi-modal fake news detection focus mainly on capturing common similar semantics between different modalities to improve detection performance. However, they ignore the extraction of inconsistent features between these modalities. The intuitive cognition way people identify a piece of fake news is generally to discover if there are inconsistent semantics among news content itself and its comments, which could be abstracted as 'comparing news image-text consistency - finding valuable comments - reasoning in-/consistency between news and comments'. Inspired by the cognitive process, we propose Human Cognition-based Consistency Inference Networks (HCCIN) to comprehensively explore consistent and inconsistent semantics for multi-modal fake news detection. Specifically, we first design cross-modal alignment layer to learn consistent semantics between textual and visual information within the multi-modal news, and then the comment clue discovery layer is devoted to ascertaining the most-concerned semantics by audiences between comments. Finally, we develop collaborative inference layer to drive news consistent semantics and the most-concerned semantics to reason and discover consistent and inconsistent information between them. Experiments on three public datasets, including Weibo, Twitter, and PHEME, reveal the superiority of our HCCIN.
AB - The existing models for multi-modal fake news detection focus mainly on capturing common similar semantics between different modalities to improve detection performance. However, they ignore the extraction of inconsistent features between these modalities. The intuitive cognition way people identify a piece of fake news is generally to discover if there are inconsistent semantics among news content itself and its comments, which could be abstracted as 'comparing news image-text consistency - finding valuable comments - reasoning in-/consistency between news and comments'. Inspired by the cognitive process, we propose Human Cognition-based Consistency Inference Networks (HCCIN) to comprehensively explore consistent and inconsistent semantics for multi-modal fake news detection. Specifically, we first design cross-modal alignment layer to learn consistent semantics between textual and visual information within the multi-modal news, and then the comment clue discovery layer is devoted to ascertaining the most-concerned semantics by audiences between comments. Finally, we develop collaborative inference layer to drive news consistent semantics and the most-concerned semantics to reason and discover consistent and inconsistent information between them. Experiments on three public datasets, including Weibo, Twitter, and PHEME, reveal the superiority of our HCCIN.
KW - Fake news detection
KW - multi-modal fusion
KW - network content security
KW - social media analysis
UR - http://www.scopus.com/inward/record.url?scp=85161029756&partnerID=8YFLogxK
U2 - 10.1109/TKDE.2023.3280555
DO - 10.1109/TKDE.2023.3280555
M3 - 文章
AN - SCOPUS:85161029756
SN - 1041-4347
VL - 36
SP - 211
EP - 225
JO - IEEE Transactions on Knowledge and Data Engineering
JF - IEEE Transactions on Knowledge and Data Engineering
IS - 1
ER -