Automatically load inverse path entities

[saumier]

When looking at the data in the app, I am however not getting “Fastnachtsbeichte” any more; this means, the reverse statements are not properly loaded on secondary items (see screenshot below). Wikidata currently has a list of 17 items where Zuckmayer is listed as author, but I am getting only links to the ones which are at the same time works performed at Schauspielhaus and thus part of the Primary Selection (light green). I also checked the data dump from the API; these statements are missing there as well.

[saumier]

Indeed there is a difference. I am able to reproduce your example, and I can explain why.

Your explanation of the primary selection and the traversal of edges is correct including the distance up to three edges away. As you pointed out, the problem lies in the reverse statements. The code in Culture In-Time automatically dereferences linked entities that are stored in Culture In-Time, up to three edges away, however the literary work "Die Fastnachtsbeichte" is not linked to any entity, forwards or backwards, in Culture In-Time.

The loading of secondary and tertiary statements works like this:

1. Find all URIs loaded with the primary entities but missing "instance" or "subclass" statements. URIs can be in subject or object position.
2. Dereference each URI
3. Repeat a second time to get tertiary statements

Here is the Wikidata documentation on how it dereferences URIs with its Linked Data Interface

The Wikidata dereferencing service returns data about a specific entity, like "Carl Zuckmayer". It appears to me that the data returned by Wikidata includes only forward linked properties along with "object" URI labels and descriptions. This is not the case for all dereferencing services. Some give reverse properties as well. Similar to the SPARQL DESCRIBE where the choice of which statements to return is determined by the server. Unfortunately the data for "Carl Zuckmayer" returned by Wikidata dereferencing service (Linked Data Interface) does not return "Die Fastnachtsbeichte", or any reverse property entities AFAIK. I am therefore concluding that the automatic loading of secondary and tertiary statements for Wikidata can only follow forward properties. In other words, even though Culture In-Time can traverse the reverse statements on secondary items, it can only load reverse statement entities if they exist in the Culture In-Time database.

I took a closer look at the turtle data https://www.wikidata.org/wiki/Special:EntityData/Q76820.ttl and by chance it does include the literary work "Die Fastnachtsbeichte" but only because it is a reference to the Goodreads author ID (see https://www.wikidata.org/wiki/Q76820) and not because it is included as ^author "Carl Zuckmayer".

To explain why "Die Fastnachtsbeichte" appeared in the past, I can point to the fact that we previously loaded Works as a Data Source. Currently, in the production environment, we are only loading Productions (primary selection) as a Data Source and all second and third edges away of the data in Culture In-Time (not the data in Wikidata).

The only way to load "Die Fastnachtsbeichte" in the current design of Culture In-Time is to create an additional source that will include "Die Fastnachtsbeichte".