Establishing Relationships Between Extracted Entities and other Custom Nodes in a Knowledge Graph

[DanielGBabel]

I am currently working on indexing my project's documentation into the Knowledge Graph. This graph includes several business domain-specific entities, and the RAG works quite effectively in this context. However, I now need to establish relationships between these entities and the corresponding SQL tables within my project, where applicable.

To achieve this, I have developed a script that processes a list of .sql schemas and constructs a schema representation in Neo4j. The schema consists of three primary node types: Database, Table, and Column. The relationships are defined as follows:

Database -> HAS_TABLE -> Table
Table -> HAS_COLUMN -> Column
Each Table node also includes a description to facilitate the creation of embeddings for future similarity searches.
And Each Column node have also a description and other properties as "type (INTEGER or DATE or CHARVAR)" and also have other relationships with other Columns and the parent table that represents IS_FOREIGN_KEY and IS_PRIMARY_KEY .

My question is: How can I identify and establish relationships between the extracted entities, document chunks, and other components within the Knowledge Graph, and my custom-created nodes (e.g., Table, Column)?

I would greatly appreciate any guidance or suggestions on how to approach this problem.

[JoshChima]

Hey Daniel.
I'm working in similar domain as you. I have tried to both create a custom prompt for graphrag & custom ddl to entity graph extractor.

Via GraphRAG Manual Prompting:
The way I went about it was to create new entity types specific to SQL => SQLTABLE, SQLCOLUMN, SQLVIEW. I also created relationship definitions that would look something like this:

Table -> HAS_COLUMN -> Column
Column -> IS_PRIMARY_KEY_OF -> Table
Column -> IS_FOREIGN_KEY_IN -> Table
Column -> IS_FOREIGN_KEY_REF_FOR -> Column
View -> CREATED_USING -> Table
View -> CREATED_USING -> View

Column, Table and Views would all contain, details about their own details like the following:
Table | View

• DDL
• Description

Column

• Data type
• Description
• Type Specific Details (TODO)
  • Unique Values: If the data is categorical, we would include the values it could have

Even with this. I find the custom solution to have more accurate and comprehensive results for defining the database structure, but I am also looking to find a way to apply my custom solution as part of the indexing engine.

All of this is still a work in progress. Maybe we could collab.

[JoshChima]

Here is the SQL Prompt I've put together. I've added some input examples too: sql-entity-extractor-prompt.py