Neo4j 5 Client Warnings due to Missing Labels along with Slower Load Performance

[jrsperry]

Using Spring Data Neo4j 7.0.0 and neo4j 5, if you have defined a node label or relationship in your schema which does not exist yet in the database you will get significantly slower loading from the db.

Here's a link to a public github project. In the "sdn_7.0.0" branch there are 3 tests which show the differences between loading.

The only difference between a Person and a PersonV2 is that PersonV2 has a relationship to a "Manager" node.

The first test (loadTest) saves 100 Person entities with no other node labels. It takes about 27 seconds to load 100 Person entities.

The second test (loadTestPersonv2MissingManagers) loads 100 PersonV2 entities (no connected managers). This test takes 36 seconds to load 100 PersonV2 entities and throws lots of warnings from the db about missing labels and relationships.

The 3rd test (loadTestWithManagers) loads 100 PersonV2 entities (no connected managers) and 1 PersonV2 with a manager. This test takes about 33 seconds to load 100 PersonV2 entities and does not throw any warnings.

In my real world use case I deal with dbs which may not have all defined entities already existing in the db. Getting slower loading is confusing to me and the amount of logs is overwhelming. In neo4j4 I had no such problem.

[meistermeier]

Thanks for reaching out. I will check the performance problem.

For the warning messages: This is something that was introduced by Neo4j itself. It has for SDN right now the very same level as a "usual warning". Thus we cannot do anything on the framework side right now.
What you could do is to add a Filter to your logging config. For logback, this would be:

<appender>
...
<filter class="ch.qos.logback.core.filter.EvaluatorFilter">
    <evaluator>
        <expression>return message.contains("relationship type is not in the database");</expression>
    </evaluator>
    <OnMismatch>NEUTRAL</OnMismatch>
    <OnMatch>DENY</OnMatch>
</filter>
</appender>

The approach above would require you to pull the Janino compiler for the default EvaluationFilter into your set of dependencies.

<dependency>
	<groupId>org.codehaus.janino</groupId>
	<artifactId>janino</artifactId>
	<version>3.1.9</version>
	<scope>test</scope>
</dependency>

[meistermeier]

TBH I cannot see a huge difference between SDN 7 and SDN 6 in this scenario. Maybe I am bad at benchmarking this, abut the numbers were about the same. I switched to Spring Boot 2.7.8. to get SDN 6.3.latest.
Maybe it is due to the logs with a mixture of single loads for those 100 entries each that will yield those log statements.

Happy to do a more directed comparison, if you have any input on what to change for a comparison.

[jrsperry]

Thanks for looking into it and for the logging advice. I was worried it was from the db and not SDN.

I'll run the tests again on my side to see if I am missing something as well with the SDN 7 vs 6 comparison. I've been struggling to come up with a more simple example that shows performance deficits between neo4j5 and neo4j4 (and to some extent SDN 7 vs SDN 6) as when I upgraded from neo4j4 to neo4j5 my projects slowed down a good amount, enough to keep me on neo4j4 for the time being.

I just reran those same tests and got pretty different results from before (much faster than 30 seconds). I'll see if I can come up with something more reproducible.

[meistermeier]

Even though this ticket is pretty, I want to respond with the latest developments.
For logging: There is now a much finer logging in place https://docs.spring.io/spring-data/neo4j/reference/appendix/index.html#logging that will get enhanced with future notification categories. This eliminate the need for custom log filter.
For mapping: With the upcoming version of 7.1.x and 7.2.x there will be a improvement when it comes to same-type references (or more complex cycles in the domain model). #2782

[spring-projects-issues]

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

[spring-projects-issues]

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.