Simpler install process for new users #305
Conversation
Jest Coverage
|
| ``` | ||
| mkdir gate-teamware | ||
| cd gate-teamware | ||
| curl -LO https://gate.ac.uk/get-teamware.sh |
There was a problem hiding this comment.
For now this is a file hosted directly on gate.ac.uk, but once we have a published release it will become a redirect to the latest GitHub release version.
| case $DEPLOY_ENV in | ||
|
|
||
| production|prod) | ||
| DJANGO_ALLOWED_HOSTS=$TEAMWARE_HOST_URL_PRODUCTION | ||
| export DJANGO_ALLOWED_HOSTS |
There was a problem hiding this comment.
The inherited process environment overrides .env for docker-compose, so we can declare DJANGO_ALLOWED_HOSTS directly in the .env but still override it if we want to for staging/test. This then means that ./deploy.sh prod is directly equivalent to a simple docker compose up -d
| @@ -15,7 +15,7 @@ if [ -f .env ]; then | |||
| fi | |||
|
|
|||
| # get current branch | |||
| BRANCH=$(git rev-parse --abbrev-ref HEAD) | |||
| BRANCH=$(git rev-parse --abbrev-ref HEAD 2>/dev/null || echo main) | |||
There was a problem hiding this comment.
Asume main (which ultimately means the published ghcr.io images) if generate-docker-env.sh is run from outside of a git clone.
| @@ -78,7 +90,7 @@ SUPERUSER_PASSWORD=${SUPERUSER_PASSWORD:-password} | |||
| # *including* the trailing slash | |||
| # | |||
| # leave blank for local images | |||
| IMAGE_REGISTRY=${IMAGE_REGISTRY:-ghcr.io/gatenlp/} | |||
| IMAGE_REGISTRY=${IMAGE_REGISTRY-ghcr.io/gatenlp/} | |||
There was a problem hiding this comment.
This and a number of other settings can legitimately be set to the empty string, so I've switched from :- (default if unset or if set to the empty string) to just - (default if unset, but if set to empty then leave empty)
| # | ||
|
|
||
| # GATE Teamware version that this script will install | ||
| DEFAULT_IMAGE_TAG=latest |
There was a problem hiding this comment.
This gets replaced by the release tag by the create-release workflow, so the script published as an artifact in a given release will install the version of Teamware corresponding to that specific release.
install/get-teamware.sh
Outdated
| recommend that public access be via secure HTTPS, for which you will need | ||
| to deploy a separate front end proxy forwarding to the GATE Teamware | ||
| application; configuration of such a proxy is beyond the scope of this quick | ||
| start script. |
There was a problem hiding this comment.
Should we include some text similar to this regarding proxy forwarding alongside the install instructions to forewarn users? Just so someone doesn't run this script and find this out right at the end?
There was a problem hiding this comment.
I've gone one better in my latest revision - I've added an optional add-on to the docker-compose stack that brings up an https reverse proxy on port 443. I don't want to make that a standard part of the stack because it only works if the machine is open to the internet on 80/443 (because it uses ACME to obtain certificates) but it's a way to mark the path of least resistance.
ianroberts
force-pushed
the
quick-start-script
branch
from
80784e3 to
0c186b6
Compare
|
This PR is becoming bigger and more complex. I noticed that we weren't passing any of the email configuration environment variables through to the backend from the compose file, so I added those; then saw we weren't even setting the "require email activation" parameter in Anyway, I think this should be more or less ready now, and I've tested it end-to-end on an AWS EC2 machine with the Caddy HTTPS reverse proxy in the mix as well. |
|
Caddy is a web server written in Go that is designed to be HTTPS-by-default with zero configuration. If you tell it that it is serving a publicly-resolvable DNS hostname then it will go out and obtain trusted certificates using ACME and keep them renewed and up to date automatically. If you tell it to serve |
|
I'm happy if you're happy, at least to make this version 1, but since I've trodden on many of the same files as #298 I'll let you merge that first and then rebase this. |
If you don't mind? I was waiting for this one to be merged, happy to do it either way round. |
|
You go ahead and finish the privacy policy first, I’ll add questions about the contact details settings to the quick start script as part of my rebase and then merge mine. |
ianroberts
force-pushed
the
quick-start-script
branch
from
13c3f6b to
33dcd5c
Compare
|
Rebased on latest dev including privacy policy changes. Could I ask for a final sanity check before merging as I've made a number of changes compared to the previous push - in particular I've tried to sort out proper quoting of values in the |
ianroberts
force-pushed
the
quick-start-script
branch
2 times, most recently
from
fb2b521 to
88b2e86
Compare
This allows the production deployment to be run directly using "docker compose up" rather than requiring the use of deploy.sh (though deploy.sh still works and is still necessary if you want to override variables for staging or test deployments)
…rom the directory if it exists
We should now properly preserve spaces etc. in multi-word env vars when re-writing the env file. I've gone with double quoting strings but backslash-escaping double quotes, dollar signs and backslashes within them. At first glance it would have been easier just to use single quotes, but there are differences between the way docker-compose parses an .env file and the way it behaves when "source"d by bash, which are easier to paper over if we use double quoted strings.
Under docker-compose rules you can escape single quotes inside single quoted strings as \', but bash has no way to put a single quote inside a single quoted string full stop. So we have to use double quotes if we want to allow for both " and ' inside the same variable value. But different parsers have different rules about other escapes inside double quoted strings:
- bash, "docker compose" (v2) and "docker-compose" (v1) all require you to escape \ as \\ and " as \"
- bash and compose v2 require escaping of $ as \$, but docker-compose v1 does not and reads a \$ "escape" as the literal backslash-dollar sequence
- bash requires escaping of ` as \` but neither version of compose requires this, reading \` as \`
However, all three parsers support ${VAR} substitution so what we actually do is:
1. declare special variables __BT__='`' and __DOLLAR__='$' at the top of the .env file
2. when we want to quote a value, we replace
- " with \"
- \ with \\
- $ with ${__DOLLAR__}
- ` with ${__BT__}
The resulting .env file reliably reads the same values under all three parsers, provided you don't try and include line breaks within a value.
…inimal additional config, rather than requiring a whole clone
…ot so much for developers as for people who want to host their own instance of a released version of Teamware
… install files as release artifacts.
…n if the Teamware instance as a whole is configured to require activation by email
ianroberts
force-pushed
the
quick-start-script
branch
from
88b2e86 to
d03d584
Compare
|
Thanks, I've done a final rebase and merged into dev |
Now we are publishing release Docker images to
ghcr.ioit is no longer strictly necessary to clone the whole gate-teamware repository in order to run a release version (or in fact a dev version) under docker-compose. All you actually need is:generate-docker-env.shto generate your initial.envfilecreate-django-db.shwhich is bind-mounted into the postgresql container to create the initial DBnginxfolder which is bind-mounted into the frontend container to reverse proxy to the back end.Everything else comes in the images that are pulled from
ghcr.io.This PR provides a "quick start" mechanism to enable this compose deployment from the public images. I've simplified the
.envgenerator so that once a suitable.envfile is in place the "production" deployment can be run purely usingdocker compose upwithout needing the separatedeploy.shscript. I've then created a "get Teamware" script that runs the user through a set of interactive prompts for the critical configuration data (admin email address, public host name, SMTP server details, etc.) and then downloads and configures a minimal compose stack with just those four items listed above, ready to be launched usingdocker compose. This was inspired by https://acme.sh, which has a bootstrapping shell script at a short URL which you download and run to download and install the rest of the tool.Initially I've hand-created the installation bundle but going forward the idea is that when we make a
vN.M.Ptag on GitHub the "get Teamware" script and associated install files bundle will be generated by an action and uploaded as GitHub release artifacts. The short URLhttps://gate.ac.uk/get-teamware.shwill then become a simple redirect to the file in the latest GitHub release, so anyone can install the latest version by making an empty dir and runningI've also moved the instructions for how to install releases out of the "developer" guide and into the main README since they're no longer specific to developers of Teamware as such.