[Notes] VLDB 2024 - KG+LLM workshop

[heathersherry]

Talk 1 – Integrating Knowledge Graph with Large Language Model: From the Perspective of Knowledge Engineering (Prof Qi)
 Introduction of KG and LLM
 KG for LLM (1) pretraining (2) as prompt (e.g., KAPING) (3) fine-tuning (4) inference (reduce the hallucination) (5) RAG (6) knowledge update or edit (7) knowledge fusion (8) knowledge validation (as benchmark) (9) future: learning of symbolic knowledge, benchmark, improve the interpretable reasoning of llm
 LLM for KG (1) entity and relation extraction (in-context learning & SFT) (2) triple generation (3) ontology matching (e.g., olala) (4) entity alignment (e.g., chatea) (5) KBQA (e.g., ELLMKGQA framework) (6) ontology reasoning (data construction: subsumption checking + instance checking) (7) KG reasoning (kg embedding is enhanced, e.g., KoPA)
 KG&LLM integration (1) knowledge service platform based on KG&LLM Integration (2) OpenKG + Tool + Application

Talk 2 Industry-level KG platform
Opportunities and Challenges
 Data driven enhancement of LLM and KG in enterprise digital scenarios (LLM only? KG enhanced LLM? LLM augmented KG? KG only?)

• Business growth
• Risk control
• Knowledge construction
   Opportunities and challenges of kg technology development
• Lack of unified knowledge modeling methods
• High cost of knowledge construction and acquisition
  Opensource works: SPG (Semantic-enhanced Programmable Graph)
   Schema design: everything as classes, unique instances
   L1-L3 development: Pay as you go, domain constraints, evolving of different data
   Subject/object type definition
   KG construction based on structured data
   Predicate semantics and logical symbols
   Event extraction based on KG construction pipeline (subgraph query)
   Implementing logic chain based on the semantic logic
   Graph leaning subgraph sampling based on GNN
  Applications:
   Event evolving KG, interpretable reasoning
   AntKnowledge Graph platform
  Future
   SPG and LLM bidirectionally driven controllable AI
   Continuously update semantic representation
   AI framework based on the OpenSPG knowledge engine
   KG are better instruction synthesizer of LLMs
  Github link of OpenSPG

Talk 3 KG enhanced LLM fine-tuning and applications
Our existing works
Knowledge Extraction
 Existing works: Domain NER, RE; continual event extraction, doc-level causality identification

• Continual RE: can learn new relations (e.g., CEAR – continual extraction for analogous relations)
  Knowledge Fusion
   Existing works: embedding based entity alignment, human in the loop, knowledge transfer (about blockchain, VLDB 2024), benchmark
  How to use KG for fine-tuning?
   KnowLA:
• 4 modules: Entity linking + LLM encoding + Knowledge mapping and injection + knowledge fusion
• 2 foundational models: llama2, alpaca2; baselines: KAPLING
• KnowLA captures more knowledge in FFN layer
   Configuration translation (the problem from theory lab)
• Workflow: candidate construction + RAG + LLM PERF
  Future Directions
   LLM PERF good!

Talk 4 OneEdit: a neural symbolic collaboratively Knowledge Editing System
 Motivation: Why knowledge editing? Fresh/conflict knowledge update and sensitive knowledge remove [picture]
 System design (3 parts): interpreter (recognize and extract triples), controller (conflict resolution, kg judgement, kg augmentation), editor (an opensource tool, + a cache for rollback)
 Conflict resolution: conflict identification + knowledge editing
 Evaluation metrics: reliability, locality (limited edit is better), portability (reverse, one-hop, sub replace) [这很奇怪，万一发生一个牵一发而动全身的修改呢？如英女王？奥运金牌更新？- answer：this is only a prototype system with 2 small kgs]
 Future work: Control Machine for Trustworthy AI

Talk 5 Leveraging LLMs few-shot learning to improve instruction drive KG construction
 Motivation: traditional methods (human crafted, e.g,. TextRunner and KnowItAll) – cannot see new entities, other (automatically RE and EE) – error propagation
 Main idea: utilize LLM to construct KG and incorporate user instruction (from CCKS 2023)

Talk 6 SPIREX: LLM-based relation extraction (medical KG)
 RNA-KG: standard bio-medical ontologies, 600K nodes, 12.5M directed edges

PS Streaming Graph Processing
 Streaming data processing
 Stream graph processing

PM
Industry Talk: Integrating GenAI with Graph: Innovations and Insights from NebulaGraph
 Structural Data RAG
 Is GraphRAG costly? No.
 GraphRAG vs KG-RAG (more ideal case, the indexing involves KG construction, …)
 NebulaGraph RAG – Enterprise KG system
Blog: - siwei.io

Paper Talk 1: Research Trends for the Interplay between Large Language Models and Knowledge Graphs
 LLM for KGs

• KG construction:
  o Ontology Creation (concept extraction, property identification, ontology alignment, text-to-ontology mapping, ontology learning)
  o Entity Extraction and Alignment
  o Relation Extraction (supervised, few-shot, zero-shot)
• KG-to-Text generation
• KG reasoning
• KG completion
• KG embedding
• KG validation (fact checking, inconsistency detection)
   KG enhanced LLM
• RAG
• GraphRAG
   LLM-KG Cooperation
• KGQA
  o Multi-hop question generation
  o Complex or multi-hop QA
  o Query generation from text
  o Querying LLMs with SPARQL
  o KG chatbots