Plugins section of virtuoso.ini gets overwritten in Docker setup

[KonradHoeffner]

Half a year ago, I reported strange behaviour when trying to deactivate plugins using the TenForce container. However I thought this was due to some Docker inexperience on my behalf and I waited for the official OpenLink image to be available with the new guide on how to set up a fully working endpoint with configuration and data.

Now that this guide is out (and much appreciated), however, the strange behaviour continues with the official Docker image openlink/virtuoso-opensource-7 as well.
When I mount a file into the Docker container and use the VIRTUOSO_INI_FILE environment parameter, some bizarre kind of merge seems to happen that uses some part of my virtuoso.ini while overwriting other parts (the plugin section) with the default values.

docker-compose.yml

services:

  virtuoso:
    image: openlink/virtuoso-opensource-7
    environment:
      - DBA_PASSWORD=${DBA_PASSWORD}
      - VIRTUOSO_INI_FILE=/ini/virtuoso.ini
    volumes:
      - ./ontology:/database/data                      
      - ./virtuoso/ini:/ini    
      - ./virtuoso/sql:/opt/virtuoso-opensource/initdb.d
      [...]

Now I run docker compose down -v and docker compose up and (there is only one container running):

$ docker exec -it `docker ps | tail -1 | cut -f1 -d " "` /bin/bash
root@e60cb3961024:/opt/virtuoso-opensource/database# tail -20 /ini/virtuoso.ini

[SPARQL]
;ExternalQuerySource            = 1
;ExternalXsltSource             = 1
DefaultGraph                   = http://www.snik.eu/montology
;ImmutableGraphs                = http://localhost:8890/dataspace
ResultSetMaxRows           = 10000
MaxQueryCostEstimationTime = 400        ; in seconds
MaxQueryExecutionTime      = 60 ; in seconds
DefaultQuery               = select distinct * where {?s ?p ?o} LIMIT 100
DeferInferenceRulesInit    = 0  ; controls inference rules loading
MaxMemInUse                = 0  ; limits the amount of memory for construct dict (0=unlimited)
;LabelInferenceName             = facets        ; Only needed when using the Faceted Browser
;PingService                    = http://rpc.pingthesemanticweb.com/

[Plugins]
LoadPath = ../hosting
Load1    = plain, meos
Load2    = plain, beo
Load3    = plain, scheos

root@e60cb3961024:/opt/virtuoso-opensource/database# tail -20 ./virtuoso.ini

[SPARQL]
;ExternalQuerySource            = 1
;ExternalXsltSource             = 1
DefaultGraph               = http://www.snik.eu/montology
;ImmutableGraphs                = http://localhost:8890/dataspace
ResultSetMaxRows           = 10000
MaxQueryCostEstimationTime = 400	; in seconds
MaxQueryExecutionTime      = 60	; in seconds
DefaultQuery               = select distinct * where {?s ?p ?o} LIMIT 100
DeferInferenceRulesInit    = 0	; controls inference rules loading
MaxMemInUse                = 0	; limits the amount of memory for construct dict (0=unlimited)
;LabelInferenceName             = facets        ; Only needed when using the Faceted Browser
;PingService                    = http://rpc.pingthesemanticweb.com/

[Plugins]
LoadPath = ../hosting
Load1    = plain, geos
Load2    = plain, proj4
Load3    = plain, shapefileio

As you can see, my changes for testing are successfully applied to the SPARQL section, but seem to be completely ignored in the Plugins section.

I find this really confusing and would be glad for some help on how to deactivate plugins or in general a pointer on which "magic behaviour" is happening when creating/copying virtuoso.ini. I would expect when I point it to a virtuoso.ini file, that it uses exactly that file and not some mixture that I have no control over.

[pkleef]

I will have a look if we can turn this off when you have a custom virtuoso.ini.

The current entrypoint script rewrites the virtuoso.ini file and rewrites the [PLUGINS] section with the actual plugins that are installed in the hosting directory. Note that Virtuoso actually has more plugins, but on a default docker image, only the ones for working with GeoSPARQL data are installed.

Out of interest, what are the meos, beo and scheo plugins, or did you just put some arbitrary names in there?

[KonradHoeffner]

Thanks for the quick reply! Sorry, I just type random names that come to mind when I debug problems like these; meo, beo, and scheo don't actually exist as plugin names. I just want to deactivate them.

As a user, I find it very hard to debug problems when there is undocumented "magic" happening in the background, especially if there are other layers involved, such as Docker and caching. At first, I was only changing the plugin section and it seemed to me as if the VIRTUOSO_INI_FILE Docker environment variable was ignored. What I found especially confusing is that if I overwrite the value of, for example, Load1 using the Docker environment parameter VIRT_PLUGINS_LOAD1='', then it does take into effect, but causes an error message, because the empty plugin does not exist.

[pkleef]

Our Virtuoso Docker Reference Guide already mentioned that the VIRT_* environment variables overrule the content of the virtuoso.ini file after it has been installed from either the VIRTUOSO_INI_FILE setting or the default virtuoso.ini file. I changed it to also include the rewriting of the [PLUGINS] section.

This allows our users to make a default virtuoso.ini template for all of their images and then tweak some details per instance.

As it is very easy to make custom docker images using our images as a template, it is possible to either install additional plugins to the image, or remove existing plugins from the image. To avoid having to make a custom virtuoso.ini in this case, the startup script will check all the .so files in the $VIRTUOSO_HOME/hosting directory and make sure they are loaded in sequence. This also makes sure that casual users do not get multiple missing plugin warnings when starting up an instance for the first time.

So if you want to remove some of the entries in the [PLUGINS] section, you can simply add the following to your environment settings in your docker-compose.yml file.

environment:
      - DBA_PASSWORD=${DBA_PASSWORD}
      - VIRTUOSO_INI_FILE=/ini/virtuoso.ini
      - VIRT_PLUGINS_LOAD1=-
      - VIRT_PLUGINS_LOAD2=-
      - VIRT_PLUGINS_LOAD3=-

or if you just want to delete the whole [PLUGINS] section you can use the special syntax:

environment:
      - DBA_PASSWORD=${DBA_PASSWORD}
      - VIRTUOSO_INI_FILE=/ini/virtuoso.ini
      - VIRT_PLUGINS_-=-

Note i have made some fixes to our documentation and Docker Image descriptions to reflect the above.

[KonradHoeffner]

Thank you for the quick help, this works perfectly!