Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Persisting to Neo4J without OGM #1893

Merged
merged 26 commits into from
Dec 13, 2024
Merged

Persisting to Neo4J without OGM #1893

merged 26 commits into from
Dec 13, 2024

Conversation

oxisto
Copy link
Member

@oxisto oxisto commented Dec 10, 2024

This PR is an attempt to build a lightweight persistence layer to Neo4J without the Neo4J OGM which causes immense problems (recursion depth, out of memory, etc.).

In the long run we might be able to get rid of the neo4j-ogm, which would allow us to re-design the node classes a little bit because we are not restricted by the OGM anymore.

  • Persist all Edge edges
  • Persist all AST nodes
  • Persist edge properties
  • Persist types + edges to nodes
  • Persist scopes + edges to nodes
  • Persist languages + edges to nodes

@oxisto oxisto changed the title neo4j without ogm Persisting to Neo4J without OGM Dec 10, 2024
Copy link

codecov bot commented Dec 10, 2024

Codecov Report

Attention: Patch coverage is 86.81818% with 29 lines in your changes missing coverage. Please review.

Project coverage is 76.71%. Comparing base (01f5873) to head (b00078d).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
...otlin/de/fraunhofer/aisec/cpg/persistence/Neo4J.kt 85.05% 13 Missing ⚠️
...tlin/de/fraunhofer/aisec/cpg/persistence/Common.kt 86.81% 5 Missing and 7 partials ⚠️
...r/aisec/cpg/graph/declarations/TupleDeclaration.kt 50.00% 1 Missing ⚠️
.../de/fraunhofer/aisec/cpg/graph/scopes/NameScope.kt 0.00% 0 Missing and 1 partial ⚠️
...tatements/expressions/InitializerListExpression.kt 0.00% 1 Missing ⚠️
...n/de/fraunhofer/aisec/cpg_vis_neo4j/Application.kt 92.30% 0 Missing and 1 partial ⚠️
Additional details and impacted files
Files with missing lines Coverage Δ
...e/fraunhofer/aisec/cpg/TranslationConfiguration.kt 91.58% <ø> (ø)
...tlin/de/fraunhofer/aisec/cpg/TranslationContext.kt 100.00% <ø> (ø)
...otlin/de/fraunhofer/aisec/cpg/TranslationResult.kt 54.76% <ø> (ø)
...kotlin/de/fraunhofer/aisec/cpg/graph/Assignment.kt 100.00% <100.00%> (ø)
...in/de/fraunhofer/aisec/cpg/graph/MermaidPrinter.kt 90.00% <100.00%> (ø)
.../main/kotlin/de/fraunhofer/aisec/cpg/graph/Node.kt 86.74% <100.00%> (+0.16%) ⬆️
...nhofer/aisec/cpg/graph/declarations/Declaration.kt 100.00% <ø> (ø)
...r/aisec/cpg/graph/declarations/FieldDeclaration.kt 78.94% <100.00%> (+1.16%) ⬆️
...isec/cpg/graph/declarations/FunctionDeclaration.kt 75.00% <ø> (+7.55%) ⬆️
...sec/cpg/graph/declarations/NamespaceDeclaration.kt 93.75% <ø> (ø)
... and 24 more

... and 9 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@oxisto oxisto marked this pull request as ready for review December 11, 2024 14:23
Copy link
Collaborator

@konradweiss konradweiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some changes and discussion is still needed here. In some places I wonder if we can use introspectionfunctionality where we now use hardcoded propertyTypes or relationships

konradweiss

This comment was marked as duplicate.

@KuechA
Copy link
Contributor

KuechA commented Dec 12, 2024

@oxisto can you add a test that #234 is fixed?

@konradweiss konradweiss self-requested a review December 12, 2024 15:05
@oxisto
Copy link
Member Author

oxisto commented Dec 13, 2024

@oxisto can you add a test that #234 is fixed?

Done

@oxisto oxisto linked an issue Dec 13, 2024 that may be closed by this pull request
@oxisto oxisto enabled auto-merge (squash) December 13, 2024 23:31
@oxisto oxisto linked an issue Dec 13, 2024 that may be closed by this pull request
@oxisto oxisto merged commit 03ceabd into main Dec 13, 2024
2 checks passed
@oxisto oxisto deleted the neo4j-without-ogm branch December 13, 2024 23:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Names of CallExpression nodes are not visible in neo4j Exception: Numeric value out of range of long
3 participants