21 remove vocabs from deleted branches

.env.example

@@ -1,6 +1,7 @@
 PORT=3000
 SECRET=YOUR_GITHUB_WEBHOOK_SECRET
 BUILD_URL=https://test.skohub.io/build
+VOCABS_URL=https://test.skohub.io/
 DOCKER_IMAGE=skohub/skohub-vocabs-docker
 DOCKER_TAG=latest
 PULL_IMAGE_SECRET=ThisIsATest

.gitignore

@@ -3,5 +3,8 @@ public/
 dist/**/*.json
 .env
 
+# Logs
+log.log
+
 # Code Editor
 .vscode/launch.json

README.md

@@ -8,15 +8,17 @@ A webhook server that allows triggering the build SKOS vocabularies to nice HTML
 
 ## Set up
 
-    $ git clone https://github.com/skohub-io/skohub-webhook.git
-    $ cd skohub-webhook
-    $ cp .env.example .env
+    git clone https://github.com/skohub-io/skohub-webhook.git
+    cd skohub-webhook
+    cp .env.example .env
+
 
 The `.env` file contains configuration details used by the static site generator and the webhook server:
 
 - `PORT`: Port the application should use
 - `SECRET`: The secret that needs to be provided when triggering the webhook
 - `BUILD_URL`: URL of the build page. This URL with a specific ID for each build can be used to retrieve information about success or errors of a triggered build. 
+- `VOCABS_URL`: URL under which the dist folder is served. Usually the same as `BUILD_URL` but without `/build`.
 - `DOCKER_IMAGE`: The docker image which should be used to build the vocabulary, defaults to `skohub/skohub-vocabs-docker`
 - `DOCKER_TAG`: The docker tag for the `DOCKER_IMAGE`, defaults to `latest`
 - `PULL_IMAGE_SECRET`: The secret needed for the `/image` endpoint to trigger the pull of new images via webhook.
@@ -41,6 +43,20 @@ The request is triggered by a GitHub Webhook when the [`docker` workflow_job](ht
 You can set up the webhook in the skohub-vocabs repo and choose "Workflow jobs" as event type.
 The secret has to match the `PULL_IMAGE_SECRET` from `.env`.
 
+### `/vocabs`
+
+Returns a list of vocabularies that are currently served, e.g.
+
+```json
+[
+	{
+		"repository": "test/test-vocabs",
+		"vocabulary": "http://localhost:8000/test/test-vocabs/heads/main/",
+		"date": "2024-02-01T08:59:36.730Z"
+	}
+]
+```
+
 ### How does it work in detail?
 
 The webhook server is started via the `docker-compose.yml` with a simple `docker compose up`.
@@ -77,13 +93,28 @@ You also might want to increase the number of `REBUILD_MAX_ATTEMPTS` (to somethi
 To rebuild all vocabularies:
 
 1. Make a backup of the dist-folder: `cp -R ./dist ./dist-backup`
-1. Make sure to have built docker image:  `docker build -t skohub-webhook .`
-1. Then mount the dist folder of the webhook container and rebuilt the vocabs: `docker run -v ./.env:/app/.env -v ./dist:/app/dist skohub-webhook:latest "npm run rebuild-vocabs"`
+1. Make sure to have a recent docker image:  `docker build -t skohub-webhook .`
+1. Then mount the dist folder and the `.env` file and rebuilt the vocabs: `docker run -v ./log.log:/app/log.log -v ./.env:/app/.env -v ./dist:/app/dist skohub-webhook:latest "npm run rebuild-vocabs"`
 
 
 To rebuild only a specific vocabulary you can provide the GitHub repository and the branch to build:
 
-`docker run -v ./.env:/app/.env -v ./dist:/app/dist skohub-webhook:latest npm run rebuild-vocabs -- test/test-vocabs main`
+`docker run -v ./log.log:/app/log.log -v ./.env:/app/.env -v ./dist:/app/dist skohub-webhook:latest npm run rebuild-vocabs -- test/test-vocabs main`
+
+## Cleaning up vocabularies
+
+From time to time you might want to clean up the `dist` folder and the build logs.
+Since we build vocabuilaries for every branch, but do not get notified when a branch is deleted, a regular cleanup is sensible.
+To clean up we provide a cleanup script that checks for still active branches on GitHub for currently served vocabularies by parsing the build logs.
+If the branch does not exist anymore the respective folder will be removed from `dist` and the build logs will be cleaned up as well.
+
+To clean up do this:
+
+1. Make a backup of the dist-folder: `cp -R ./dist ./dist-backup`
+1. Make sure to have a recent docker image:  `docker build -t skohub-webhook .`
+1. Do a dry run first and check the output: `docker run -v ./log.log:/app/log.log -v ./.env:/app/.env -v ./dist:/app/dist skohub-webhook:latest "npm run cleanup-dist:dry"`
+1. Cleanup, if everything looks ok: `docker run -v ./log.log:/app/log.log -v ./.env:/app/.env -v ./dist:/app/dist skohub-webhook:latest "npm run cleanup-dist"`
+
 
 ## Connecting to our webhook server