which SHACL validators to try?

[VladimirAlexiev]

Requirements:

• must: SPARQLConstraint
• opt: fast and, or, not
• opt: sh:dataGraph so we can validate various stored models (consider options for using SHACL on graphs #140)
• opt: incremental validation

Let's define which validation engines to try.
We now have a pretty complete list at https://github.com/w3c-cg/awesome-semantic-shapes#readme and several contributors add to it often.

Here's a proposal:

• TopQuadrant API (Java & Jena) because it has been tested to date
  • ValiMate (part of the DNV suite) uses TopQuadrant API, so doesn't need to be tested separately
• Jena SHACL (Java) because it seems to have more recent development than TQ
• rdf4j ShaclSail (Java) because it can do incremental validation, and is used by GraphDB: see supported SHACL features
• pySHACL (Python) because a lot of TSOs/Power Engineers use Python, claims to have the most complete coverage of SHACL features, used by ModShape
• DataTreehouse/maplib (Rust) because it's fast for in-memory. However, it's not open source and DataTreehouse didn't agree to provide it for evaluation (Oct 2024)
• https://rudof-project.github.io/rudof/: a library and CLI that implements Shape Expressions, SHACL, DCTAP and other technologies in the RDF ecosystem (by the same guy who made SHACLEX, the first SHACL+SHEX implementation). It is implemented in Rust and it also provides Python bindings.

How about:

• JavaScript implementations?
• Shaclex, which implements SHACL and SHEX validation (don't know if that's relevant)

After we agree on the list, we need to research and list the limitations of every implementation.
This may eliminate some candidates.

@HarisVranaj please attach the presentation you showed 2d ago (I hope it's not confidential).
@griddigit-ci and @Sveino please comment on the proposal above, and I'll correct the list

[Sveino]

I agree on the requirement. First of all, we would like to have the UML/information model so that we really only need UML restriction. However, the world is more complicated. To avoid to have very technical UML/information model we will use a logical description of the constraints. This does not really need to be processes as is, but can be converted to relevant execution. This should be the primary motivation for not including SPARQL. Secondary is that we want to have engines that is optimised to execute well known constraints pattern.
So our primary test of the SHACL validation engines is to test our SHACL that we are applying rules that are not wrong understanding or bias to a particularly implementation.
I agree with the priorities and the argument for picking them. If we should add any addition, I would considered pySHACL. The reason for this is that a lot of TSOs are start using Python for Power Engineers. In addition Nick Car is a core developer. They have also boosted that they have the most complete coverage of the SHACL rules.

[HarisVranaj]

I have some suggestion from Erik for benchmarks in SHACL/SPARQL validator.

https://github.com/oxigraph/oxigraph/blob/main/bench/README.md
https://github.com/ad-freiburg/qlever/wiki/QLever-performance-evaluation-and-comparison-to-other-SPARQL-engines
Oxigraph is now optimised for memory usage (no longer using the rocksdb engine when using in memory) which on Erik's machine is 4 times faster then earlier versions (as this is including unzipping the file, real performance will even be better).

[VladimirAlexiev]

@HarisVranaj but do Oxigraph and QLever have SHACL implementations?
Please post links so I can include in https://github.com/VladimirAlexiev/awesome-semantic-shapes#shacl-validators and thereon to https://github.com/w3c-cg/awesome-semantic-shapes

[VladimirAlexiev]

Note to self: https://mail.google.com/mail/u/0/#sent/QgrcJHsTgsbXhdCJwNqzTbwQHVhdRXDHtBB asked Treehouse for access to maplib SHACL.

[VladimirAlexiev]

@Sveino points out that rdf4j 5.0.0 and 5.0.3 have some SHACL improvements:

• Support for sh:closed eclipse-rdf4j/rdf4j#1112
• SHACL - sh:equals, sh:lessThan, sh:lessThanEquals, sh:disjoint eclipse-rdf4j/rdf4j#4770
• SHACL - Support sh:name, sh:description, sh:defaultValue, sh:order, sh:group eclipse-rdf4j/rdf4j#4864
• SHACL - Improve logging eclipse-rdf4j/rdf4j#5027
• SHACL - Reduce duplicate use of Unique plan node eclipse-rdf4j/rdf4j#5028
• Allow configuration of threshold for using SPARQL validation approach when validation sh:maxCount eclipse-rdf4j/rdf4j#5108
• SHACL - sh:pattern validation results can differ between SPARQL vs. transactional validation eclipse-rdf4j/rdf4j#5112
• SHACL - Improve log message for internal exceptions eclipse-rdf4j/rdf4j#5113
• SHACL - Correctly handle missing values when validation SPARQL constraints eclipse-rdf4j/rdf4j#5114
• SHACL - bug where sh:class could cause missing validation results when nested sh:property and sh:node eclipse-rdf4j/rdf4j#5115
• GH-5130 sh:oneOrMorePath, sh:zeroOrMorePath, sh:zeroOrOnePath eclipse-rdf4j/rdf4j#5131
• SHACL - maxCount > 1 is slow for bulk validation eclipse-rdf4j/rdf4j#5153

And more are planned to be completed by 5.0.3 is released

• SHACL: More descriptive message when a shape is both a sh:NodeShape and a sh:PropertyShape eclipse-rdf4j/rdf4j#4289
• SHACL - Possible NullPointerException when comparing two Select plan nodes eclipse-rdf4j/rdf4j#4313
• SHACL - support binding query result variables in the sh:resultMessage when using SPARQL Constraint eclipse-rdf4j/rdf4j#4685
• SHACL - SPARQL Constraint and SPARQL Target should support variables with either of $ or ? eclipse-rdf4j/rdf4j#4688
• SHACL - Support the entire list of prioritized rules for which values should be used in the Validation report for SPARQL Constraint eclipse-rdf4j/rdf4j#4689
• SHACL - validation of sh:qualifiedMaxCount may sometimes not when a statement matching the path is added eclipse-rdf4j/rdf4j#5139

GraphDB will upgrade to rdf4j 5.0 at the end of the year.

[griddigit-ci]

When I tried pySHACL back in Jan and tried to package ModShape I has troubles. I was having performance issues. I was in touch with Nick at that time, there might be solutions, but I didn't have time to clean that up.

[HarisVranaj]

https://github.com/ad-freiburg/qlever
https://github.com/oxigraph/oxigraph/tree/main

[VladimirAlexiev]

@HarisVranaj Do Qlever and OxiGraph support SHACL? Please post links to documentation

[hmottestad]

I'm also working on supporting the last of the SHACL path expressions, and this should be included in RDF4J 5.1.0 or 5.2.0: eclipse-rdf4j/rdf4j#5131

I can also advertise that the RDF4J SHACL implementation supports incremental validation. If you have a large database and want to make a small change to your data, then the RDF4J SHACL engine will analyse your changes and only validate the affected target nodes.

[Sveino]

@hmottestad Very good. Incremental or difference validation is extremely relevant since we have a lot of SHACL rules that goes across multiple objects. The full graph is getting very big, and the changes are very limited. We have included the possiblity to exchange differences since 2005 using CIMXML/ RDFXML. We are not looking into how we can use JSON-LD to exchange this. See #53

[hmottestad]

RDF4J 5 has support for JSON-LD 1.1 with a customised version of Titanium JSON-LD that is considerably faster than the stock implementation that Jena is using.

I saw you were talking about DCAT, is your projected related to Datakatalogen på any chance?

[Sveino]

I like fast code :-)
The use of DCAT has two purposes. One is providing the header information on the dataset/named graph. We expect the same information to be linked to a Catalog so that the dataset/named graph can be found. So second purpose is to support data catalog (Datakatalog).

[HarisVranaj]

Reply from Erik. "They do not support it out of the box, only SPARQL, for them, SHACL needs to be translated into SPARQL
this one does
https://github.com/DataTreehouse/maplib
, they say python, but it's actually written in Rust with a Python API, but can be used as basis to create full app in Rust. "
@Sveino can you give access to him.

[Sveino]

@HarisVranaj I am not able to give access to DataTreehouse Github, if that is what you wanted.

[HarisVranaj]

nono to this repository.

[VladimirAlexiev]

The repo is public, so Erik can post and comment

[VladimirAlexiev]

I added https://github.com/w3c-cg/awesome-semantic-shapes#readme and https://rudof-project.github.io/rudof/ to the top.