jena-geosparql: issues when building spatial index from assembler

[Aklakan]

Version

5.5.0, 5.6.0-SNAPSHOT

What happened?

What works: Loading an existing spatial.index file and rebuilding the spatial index using the Fuseki Mod UI/API.

What fails: If the spatial.index file is absent, it should be created on Fuseki server start - via the assembler configuration - but there is weird behavior going on:

• According to https://www.mail-archive.com/users@jena.apache.org/msg21183.html the spatial index is created but not active on the dataset.
• When trying to reproduce the issue on my setup, the spatial index setup created 64 spatial index files with name pattern spatial.index-{0..63} with varying sizes until a final ordinary spatial.index file was created. Unclear whether no index or a random one eventually becomes the active one. It turned out that this was an issue of my own docker setup and unrelated to Jena.

Saving a spatial index creates a temporary spatial-index-{number} file which only on success should overwrite the final spatial.index file - this is aimed to protect against (a) corrupted saves such as when shutting down a docker container while a write is active (b) concurrent recreation of the spatial index (should not really happen because only one indexing task can be active per dataset context).

So temporary spatial index files are not unexpected but it seems that there is a bugged loop on some level involved - could be a mix of endpoints, CPUs or (named) graphs.

Relevant output and stacktrace

Are you interested in making a pull request?

Yes

[Aklakan]

Quick update: As for the additional spatial-index files, they are created because of a restart of our docker container. It's strange that it exits with code 0 and that there is no stack trace. It seems as if it crashes either while writing the temp file or while moving it to the final location.

Perhaps its just an OOM issue on our setup and unrelated to report of the generated index not being active.

our-fuseki-staging-1  | 12:38:32 INFO  SpatialIndexLib :: Building Spatial Index - Started                                                                                                        
our-fuseki-staging-1  | 12:53:49 INFO  SpatialIndexLib :: Building Spatial Index - Completed                                                                                                      
our-fuseki-staging-1  | 12:53:49 INFO  SpatialIndexIoKryo :: Saving Spatial Index - Started: /run/databases/tdb2/ourdb/spatial.index
our-fuseki-staging-1 exited with code 0                                                                                                                                                           
(next log entries are about restarting fuseki and rebuilding the spatial index)

[Aklakan]

Ok, the issue with the multiple failed index creations was simply that a configured health check pretty consistently timed out while writing out the spatial index 🤦
With increased HEALTHCHECK --start-period=60m setting everything now works for me as it should - the created index is immediately ready for use. Well, good to know that this turned out to be NOT a Jena issue.

We use slightly different options than those in the mail - namely: no inference, absolute paths, another srs and union default graph.
I wonder whether inferences need updates in addition to the spatial index. Will check.

<#geoDS>
    geosparql:inference            false ;
    geosparql:srsUri <http://www.opengis.net/def/crs/EPSG/0/4326> ;


<#baseDS> rdf:type tdb2:DatasetTDB2 ;
    tdb2:location "/run/databases/tdb2/ourdb/" ;
    tdb2:unionDefaultGraph true ;

[Aklakan]

So far I was not able to reproduce the issue reported in the mail that when using the spatial indexer, the recomputed index would only be active after a restart of Fuseki.

• Perhaps its really an issue when using the default graph - all my experiments used named graphs (see screencast below).
• The spatial indexer UI/API currently does not support reloading an externally modified spatial.index file - a reload action/button could be added if needed.

A screencast of the issue and the used query would be helpful.

Notes:

• Setting geosparql:inference true indeed materializes inferred RDFS data during startup. Because materialization alters data, this feature was not relevant to us and we did not look much into it. However, I don't see how this would conflict with the spatial indexer UI/API that would just operate on the materialized data.
• The screencast shows that replacing the index with fewer graphs - as expected - is reflected on the next query without Fuseki restart. Indexing all graphs later also works as expected without restart (not part of the screencast because indexing took very long).

jena-spatial-index-rebuild2.mp4

For reference, I used yasgui for visualization and this query that selects 1000 geometries from each graph in the given polygon:

PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX geof: <http://www.opengis.net/def/function/geosparql/>
PREFIX spatial: <http://jena.apache.org/spatial#>
PREFIX geo: <http://www.opengis.net/ont/geosparql#>

SELECT ?graph ?geom ?geomColor ?geomTooltip (?geomTooltip AS ?geomLabel) {
  VALUES ?polygon {
    "POLYGON ((6.475457372470345 51.7037981703763, 6.475457372470346 51.2481032780451, 7.544268491722988 51.24810327804511, 7.544268491722988 51.70379817037631, 6.475457372470345 51.7037981703763))"^^geo:wktLiteral
  }
  GRAPH ?graph {}

  LATERAL {  
    SELECT * {
      GRAPH ?graph {
        ?x spatial:intersectBoxGeom(?polygon) .
        ?x geo:hasGeometry/geo:asWKT ?geom .
        FILTER(geof:sfIntersects(?geom, ?polygon))
      }
    } LIMIT 1000
  }
  
  # Color geometries based on the graph that contains them
  BIND (concat('#', substr(sha1(str(?graph)), 1, 6)) AS ?geomColor)
  
  BIND(strdt('Entity <a href="' + STR(?x) + '">' + STR(?x) + '</a> from graph ' + STR(?graph), rdf:HTML) as ?geomTooltip)
}

[scs-eb]

We use slightly different options than those in the mail [...]

We used the defaults from the documentation.
I will experiment with the different options mentioned after this post just to be sure.

Perhaps its really an issue when using the default graph [...]

Sorry I didn't mentioned this in the email originally. We are currently using the default graph, so it may be possible.
I will experiment with a named graph after this post.

A screencast of the issue and the used query would be helpful.

As requested, I made two screencasts :

1. Demonstrating the issue : jena-spatial-index-rebuild.mp4
2. Demonstrating after the restart, everything is fine : jena-spatial-index-rebuild-after-restart.mp4

jena-spatial-index-rebuild.mp4

jena-spatial-index-rebuild-after-restart.mp4

I also added the query and the sample data that was used in the videos :

query.rq

PREFIX geo: <http://www.opengis.net/ont/geosparql#>
PREFIX ogcsf: <http://www.opengis.net/ont/sf#>

SELECT *
WHERE {
  ?s a ogcsf:Point .
  ?s geo:sfWithin <urn:test:geosparql#geoFrance> .
} 

data.ttl

inspired greatly from : https://github.com/zazuko/fuseki-geosparql/blob/main/tests/geosparql/data.nt

@prefix : <urn:test:geosparql#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix geo: <http://www.opengis.net/ont/geosparql#> .
@prefix ogcsf: <http://www.opengis.net/ont/sf#>.

# France (polygon)

## Feature
:France a geo:Feature; 
	rdfs:label "France";
	geo:hasGeometry :geoFrance.

## Geometry
:geoFrance a ogcsf:Polygon ;
	geo:asWKT "POLYGON((8.24127303627454 48.97794887560541,5.231019130024541 49.750647190315945,2.440491786274541 51.14961841626458,1.5176402237745412 50.87313049536895,1.5011607315870412 50.21336829946772,0.9133921768995412 49.99845644178497,0.21576034096204122 49.82512350296763,-0.2621449324754588 49.33718921868228,-1.8881214949754588 49.82157960827845,-2.085875401225459 49.394425976491604,-1.5365589949754588 48.73217609928656,-3.140562901225459 48.891343313415014,-4.942320713725459 48.630624713300485,-4.766539463725459 48.07577006394907,-4.371031651225459 47.75177459783237,-3.096617588725459 47.21722096602976,-1.5805043074754588 45.61162012205615,-1.7782582137254588 43.60971468750438,-1.8881214949754588 43.290685000073935,0.19928084877454122 42.61524836026427,0.8145152237745412 42.7928648459883,2.704163661274541 42.24222710444366,3.253480067524541 42.420905523667386,3.561097255024541 43.14657273329965,4.659730067524541 43.08241351640007,6.549378505024541 42.501955158864604,7.208558192524541 42.95389331205366,8.65875350502454 41.324731196577005,9.66949569252454 41.423662015262664,10.02105819252454 42.88953234551158,9.31793319252454 43.46635957827632,7.911683192524541 43.08241351640007,7.516175380024541 43.784463573801766,7.735901942524541 44.16396575789822,7.054749598774541 44.22698031120467,6.944886317524541 44.61929440132649,7.098694911274541 44.75988294318899,6.791077723774541 44.97789792588031,6.725159755024541 45.164110654364634,7.142640223774541 45.2724545718037,7.208558192524541 45.45770672352913,6.835023036274541 45.74978203327899,7.054749598774541 45.94874696966964,6.791077723774541 46.37487002615409,6.285706630024541 46.40518111104491,6.175843348774541 46.20785813977869,6.044007411274541 46.480885182821844,6.527405848774541 46.9178949511633,7.054749598774541 47.30669106481971,6.835023036274541 47.366253803971325,7.076722255024541 47.529704967777825,7.252503505024541 47.45547195118154,7.604066005024541 47.48517774971194,7.538148036274541 47.67785626638349,7.604066005024541 48.090448939567736,7.845765223774541 48.58704000495766,8.24127303627454 48.97794887560541))"^^geo:wktLiteral  .

# Strasbourg (point)

## Feature
:Strasbourg a geo:Feature; 
	rdfs:label "Strasbourg";
	geo:hasGeometry :geoStrasbourg.

## Geometry
:geoStrasbourg a ogcsf:Point ;
	geo:asWKT "POINT(7.7510315861075085 48.581903974563204)"^^geo:wktLiteral .

# Bern (point)

## Feature
:Bern a geo:Feature; 
	rdfs:label "Bern";
	geo:hasGeometry :geoBern.

## Geometry
:geoBern a ogcsf:Point ;
	geo:asWKT "POINT(7.448102905878211 46.945410623891554)"^^geo:wktLiteral .

[Aklakan]

Thanks for the detailed report. I could reproduce the issue and it turned out to be related to the inferences and not to the spatial index / spatial indexer UI/API. Restarting the server with inferences enabled adds geo:Geometry types which impact the execution.

Simply addinggeo:Geometry types to your input data makes you query work immediately (no restart or spatial index needed):

:geoStrasbourg a ogcsf:Point, geo:Geometry ; .

:geoFrance a ogcsf:Polygon, geo:Geometry ; .

It feels counter-intuitive that the presence of a geo:asWKT or geo:asGML is not sufficient - perhaps the requirement for geo:Geometry types can be lifted - if it is in accordance with the spec (need to check first).

Also, the way your query is executed does not use the spatial index at all:

  ?s a ogcsf:Point . # Here, ?s is bound first.

  # Here both sides of geo:sfWithin are bound.
  # Query execution fetches the geometries of both sides and tests for the relation (within).
  # No spatial index access is involved.
  ?s geo:sfWithin <urn:test:geosparql#geoFrance> .

Detailed analysis:

• geo:sfWithin goes to GenericPropertyFunction.queryRewrite

jena/jena-geosparql/src/main/java/org/apache/jena/geosparql/geo/topological/GenericPropertyFunction.java

Line 100 in 968c430

Boolean isPositiveResult = queryRewrite(graph, subject, predicate, object, queryRewriteIndex);

• which then checks for the presence of (among other things) a geo:Geometry type:

jena/jena-geosparql/src/main/java/org/apache/jena/geosparql/geo/topological/SpatialObjectGeometryLiteral.java

Line 120 in 968c430

} else if (graph.contains(targetSpatialObject, RDF.type.asNode(), Geo.GEOMETRY_NODE)) {

That said, to me it seems that the dataset-level queryRewriteIndex is susceptible to going out of sync under updates (I didn't write this part of the code). I wonder if this index could be removed (without significant performance impact) or at least moved to request execution level. Perhaps this is a separate issue though. As I read the code, queryRewriteIndex is per execution context.

[Aklakan]

A possible small bug I noticed in your screencast: The checkbox Replace index with selected graphs is checked, but no graph is selected - this should replace the index with a new index of 0 selected graphs. But the status message says that 1 graph was indexed.

In my screen cast, I deliberately replaced an existing index with a selection of fewer graphs such that data disappears from the map view. So it suggests that the corner case of replacement with an empty selection is not handled correctly.

[Aklakan]

I created #3480 that would make your reported query work without the need to add extra geometry types or enable inferencing.

The reason why inferencing is enabled in my spatial indexer documentation draft at apache/jena-site#213 is because I was working on aligning it with the existing geosparql assembler documentation where it is also enabled. Good that things are a bit clearer now.

[scs-eb]

I'm still working on completing the homework I mentioned in my previous post, mainly to clarify my understanding. So I don't have much to add, however, I wanted to reply to some of the comments :

Simply adding geo:Geometry types to your input data makes you query work immediately (no restart or spatial index needed):
I created #3480 that would make your reported query work without the need to add extra geometry types or enable inferencing.

This is great news and wow that was quick, I was still reading the comments when I saw that a PR was created. Really good job! 🥇

Also, the way your query is executed does not use the spatial index at all:

🤦 I was sure it was using it because of the restart. I will add this as an homework on my end to understand when the spatial index is involved in a query, I believe this will be useful later on.

A possible small bug I noticed in your screencast: The checkbox Replace index with selected graphs is checked, but no graph is selected [...]

🤦 Looks like a misunderstanding from my end. I checked it believing this would be better to replace the initial empty spatial index. It didn't impact the indexing though, but this is not planned for this usage as I could see by your example.