yaml configurations, external rdf files support

[jarno-knaw]

Added YAML configuration file support

In the data folder, instead of iterating through all .ttl and .config files, the application now checks for all yaml files. These yaml files can contain configuration as follows:

config:
  type: file
  location: countries.config
source:
  type: file
  location: countries.ttl

Vocabularies can also be loaded from an external source. This applies both to the RDF file and the config. In order to do so, specify the type as fetch. When no filetype is clear from the url (if it ends in something other than the normal file extension), it can be explicitly stated as well:

config:
  type: fetch
  location: https://raw.githubusercontent.com/CLARIAH/skosmos/main/data/countries.config.example
source:
  type: fetch
  location: https://raw.githubusercontent.com/CLARIAH/skosmos/main/data/countries.ttl.example
  format: ttl
    # It is also possible to specify additional headers, for other kinds of authentication or content negotiation:
  headers:
    Accept: application/turtle

Getting files from private repositories

GitHub

When retrieving files from a private GitHub repository, it is also possible to specify an access token. Also note the alwaysRefresh setting on the config, which is used to make the application import this dataset again even if it is already imported into the application, to update it:

config:
  alwaysRefresh: Yes
  type: fetch
  location: https://raw.githubusercontent.com/organization/private-repository/main/vocab.config
  auth:
    type: github
    token: your_token_here
source:
  type: fetch
  location: https://raw.githubusercontent.com/organization/private-repository/main/vocab.trig
  auth:
    type: github
    token: your_token_here

GitLab

Getting a file from a private GitLab repository uses the URL where the file can be found when browsing the GitLab site and navigating through the code to the file you need. The URL should be similar to the one in the example below. The token you provide must have access to the Repository Files API.

source:
  type: fetch
  location: https://your-gitlab-domain.com/org/repo-name/-/blob/branch-name/path/to/vocab.trig
  auth:
    type: gitlab
    token: your_token_here