Skip to content

Commit

Permalink
DCJ-495: New Cert documentation and configuration render script (#2621)
Browse files Browse the repository at this point in the history
  • Loading branch information
rushtong authored Jul 11, 2024
1 parent 23186ad commit 382870e
Show file tree
Hide file tree
Showing 4 changed files with 118 additions and 16 deletions.
24 changes: 13 additions & 11 deletions DEVNOTES.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,30 +12,32 @@ nvm install 16
npm install
```

3. Install configs for an environment. This example is for the perf environment, but you can use values from any environment by looking at the deployed configs in https://duos-k8s.dsde-{%ENV%}.broadinstitute.org/config.json where {%ENV%} is any of `dev`, `staging`, `alpha`, or `prod`
Remember to set the `env` value appropriately. We use `local` for running via npm, but under docker, we use a real env like `dev`
3. Install configs for an environment. This example is for the `alpha` environment, but you can use values from any environment by looking at the deployed configs in https://duos-k8s.dsde-{%ENV%}.broadinstitute.org/config.json where {%ENV%} is any of `dev`, `staging`, `alpha`, or `prod`
Remember to set the `env` value appropriately. We use `local` for running via npm, but under docker, we use a real env like `dev`.
The installation steps outlined in this step can also be completed using the [render-configs.sh](scripts/render-configs.sh) script which can generate all required
files for local development.
```
cp config/alpha.json public/config.json
```

Ensure that your `/etc/hosts` file has an entry for `local.broadinstitute.org`
Ensure that your `/etc/hosts` file has an entry for `local.dsde-dev.broadinstitute.org`
```properties
127.0.0.1 local.broadinstitute.org
127.0.0.1 local.dsde-dev.broadinstitute.org
```

Download cert files from vault (requires vault access - see [DUOS team members](https://github.com/orgs/DataBiosphere/teams/duos) for more specifics):
Download cert files from dev project (requires access to correct project - see [DUOS team members](https://github.com/orgs/DataBiosphere/teams/duos) for more specifics). Cert files are regenerated on a 3-month rotation so these will need to be updated when they are expired:
```shell
vault login -method=github token=$(cat ~/.github-token)
vault read --format=json <vault path>/server.key | jq -r .data.value > server.key
vault read --format=json <vault path>/server.crt | jq -r .data.value > server.crt
vault read --format=json <vault path>/ca-bundle.crt | jq -r .data.chain > ca-bundle.crt
gcloud container clusters get-credentials --zone us-central1-a --project <project> terra-dev
kubectl -n local-dev get secrets local-dev-cert -o 'go-template={{ index .data "tls.crt" | base64decode }}' > server.crt
kubectl -n local-dev get secrets local-dev-cert -o 'go-template={{ index .data "tls.key" | base64decode }}' > server.key
kubectl -n local-dev get configmaps kube-root-ca.crt -o 'go-template={{ index .data "ca.crt" }}' > ca-bundle.crt
```

Create a `site.conf` file in the project root directory using https://github.com/broadinstitute/terra-helmfile/blob/master/charts/duos/templates/_site.conf.tpl as a model.

Create a local environment file, `.env.local`
```properties
HOST=local.broadinstitute.org
HOST=local.dsde-dev.broadinstitute.org
HTTPS=true
SSL_CRT_FILE=server.crt
SSL_KEY_FILE=server.key
Expand Down Expand Up @@ -76,7 +78,7 @@ run in GitHub Actions. Create a `cypress.env.json` file in the root of your
local repo that looks like this:
```json
{
"baseUrl": "https://local.broadinstitute.org:3000/"
"baseUrl": "https://local.dsde-dev,broadinstitute.org:3000/"
}
```
Cypress will use these values in `cypress.config.js` and `cypress/support/commands.js`
Expand Down
4 changes: 2 additions & 2 deletions TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,15 +16,15 @@ npm test
Create a .local.env with the following

```
HOST=local.broadinstitute.org
HOST=local.dsde-dev.broadinstitute.org
HTTPS=true
SSL_CRT_FILE=server.crt
SSL_KEY_FILE=server.key
```

Update `baseUrl` in `cypress.config.js` and `url` in `cypress/support/commands.js` to
```
https://local.broadinstitute.org:3000
https://local.dsde-dev.broadinstitute.org:3000
```

Set `env` to `local` in `config.json`
Expand Down
6 changes: 3 additions & 3 deletions docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ services:
proxy:
image: us.gcr.io/broad-dsp-gcr-public/httpd-terra-proxy:v0.1.16
container_name: duos-proxy
hostname: local.broadinstitute.org
hostname: local.dsde-dev.broadinstitute.org
links:
- app:app
ports:
Expand All @@ -48,7 +48,7 @@ services:
OIDC_CLAIM: Require all granted
PROXY_PATH: /
PROXY_URL: http://app:8080/
SERVER_NAME: local.broadinstitute.org
SERVER_NAME_INT: local.broadinstitute.org
SERVER_NAME: local.dsde-dev.broadinstitute.org
SERVER_NAME_INT: local.dsde-dev.broadinstitute.org
ENABLE_TCELL: 'no'
restart: always
100 changes: 100 additions & 0 deletions scripts/render-configs.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
#!/bin/bash
# Populates configurations necessary for local development.
# Certs are regenerated on a 3-month rotation so this script is optimized for that task.
# You MUST be on the Broad VPN
# You MUST have jq, gcloud and kubectl installed to run this script.
# You MUST authenticate via gcloud
#
# See usage section below for more details. All arguments are optional.

set -eu
set -o pipefail

usage() {
cat <<EOF
Usage: $0 [OPTION]...
Generate cert files for local development
--project PROJECT Google project where cert files are stored
--env ENV Write an .env.local file to project root. true|false. Defaults to false
--config CONFIG Write a config.json file in public. true|false. Defaults to false
--help Display this help and exit
EOF
exit 0
}

error() {
echo "ERROR: $1" >&2
exit 1
}

# default values that may be overridden by command line arguments
PROJECT="broad-dsde-dev"
ENV="false"
CONFIG="false"

parse_cli_args() {
while [ $# -gt 0 ]; do
case "$1" in
--project)
PROJECT=$2
shift 2
;;
--env)
ENV=$2
shift 2
;;
--config)
CONFIG=$2
shift 2
;;
--help)
usage
;;
*)
error "Unknown option: $1. Try --help to see a list of all options."
;;
esac
done
}

auth_gcloud() {
echo "Getting cluster credentials"
gcloud container clusters get-credentials --zone us-central1-a --project "$PROJECT" terra-dev
}

write_certs() {
echo "Writing cert files"
kubectl -n local-dev get secrets local-dev-cert -o 'go-template={{ index .data "tls.crt" | base64decode }}' > ../server.crt
kubectl -n local-dev get secrets local-dev-cert -o 'go-template={{ index .data "tls.key" | base64decode }}' > ../server.key
kubectl -n local-dev get configmaps kube-root-ca.crt -o 'go-template={{ index .data "ca.crt" }}' > ../ca-bundle.crt
}

write_env() {
echo "Generating .env.local file"
echo "
HOST=local.dsde-dev.broadinstitute.org
HTTPS=true
SSL_CRT_FILE=server.crt
SSL_KEY_FILE=server.key" > ../.env.local
}

write_config() {
echo "Generating public/config.json file"
JSON=$(curl https://duos-k8s.dsde-dev.broadinstitute.org/config.json)
echo "$JSON" > ../public/config.json
jq '.env = "local"' ../public/config.json > /dev/null
jq '.tag = "dev"' ../public/config.json > /dev/null
jq '.hash = "dev"' ../public/config.json > /dev/null
}

parse_cli_args "$@"
auth_gcloud
write_certs
if [ "$ENV" == "true" ]
then
write_env
fi
if [ "$CONFIG" == "true" ]
then
write_config
fi

0 comments on commit 382870e

Please sign in to comment.