Logic-Aware Knowledge Graph Reasoning for Structural Sparsity under Large Language Model Supervision

Yudai Pan, Jiajie Hong, Tianzhe Zhao, Lingyun Song, Jun Liu, Xuequn Shang

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Knowledge Graph (KG) reasoning aims to predict missing entities in incomplete triples, which requires adequate structural information to derive accurate embeddings. However, KGs in the real world are not as dense as the idealized benchmarks, where sparse graph structures restrict the comprehensive structural information for superior performance. Although the logical semantics in KGs shows its potential in alleviating the impact of structural sparsity, there still exist some challenges. The deficient supervision and the semantic gap of logic make it difficult to introduce logical semantics in sparse KG reasoning. To this end, we propose a novel KG reasoning approach LoLLM injecting logic with the supervised information supplied by the Large Language Model (LLM), which is proved to be effective in evaluating and scoring. Firstly, LoLLM derives structural embeddings employing a graph convolutional network (GCN) with relation-aware and triple-aware attention. LoLLM secondly constructs reasoning paths instantiated from the first-order logic rules extracted from sparse KGs, and injects the logical semantics by a designed LLM-enhanced tuning strategy. We propose a textual loss (TL) and a logical loss (LL) in the optimization and obtain logical tuning embeddings of KG in this process. Finally, LoLLM fuses structural embeddings from the GCN and logical tuning embeddings from the LLM-enhanced tuning for scoring and incomplete triple prediction. Extensive experiments on two sparse KGs and a benchmark show that LoLLM outperforms state-of-the-art structure-based and Language Model (LM)-augmented baselines. Moreover, the logic rules with corresponding confidences provide explicit explanations as an interpretable paradigm.

Original languageEnglish
Title of host publicationWWW 2025 - Proceedings of the ACM Web Conference
PublisherAssociation for Computing Machinery, Inc
Pages4531-4542
Number of pages12
ISBN (Electronic)9798400712746
DOIs
StatePublished - 28 Apr 2025
Event34th ACM Web Conference, WWW 2025 - Sydney, Australia
Duration: 28 Apr 20252 May 2025

Publication series

NameWWW 2025 - Proceedings of the ACM Web Conference

Conference

Conference34th ACM Web Conference, WWW 2025
Country/TerritoryAustralia
CitySydney
Period28/04/252/05/25

Keywords

  • First-order Logic
  • Knowledge Graph
  • LLM-enhanced Tuning
  • Sparse Knowledge Graph Reasoning

Fingerprint

Dive into the research topics of 'Logic-Aware Knowledge Graph Reasoning for Structural Sparsity under Large Language Model Supervision'. Together they form a unique fingerprint.

Cite this