Skip to content

Commit

Permalink
Merge branch 'main' into forms-960-idp-refactor
Browse files Browse the repository at this point in the history
  • Loading branch information
usingtechnology committed Feb 28, 2024
2 parents 7091e05 + e15c93a commit 7c610cd
Show file tree
Hide file tree
Showing 38 changed files with 454 additions and 133 deletions.
5 changes: 5 additions & 0 deletions .devcontainer/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,8 @@ FROM node:${VARIANT}

# not much in here, could acheive this another way for sure...
# but this allows us a prepared place to add other things to the container OS.

RUN apt-get update && \
DEBIAN_FRONTEND=noninteractive apt-get install -y \
# For interactive git rebases
vim
53 changes: 31 additions & 22 deletions .devcontainer/README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
# CHEFS Development with Dev Container
The following guide will get you up and running and developing/debugging CHEFS as quickly as possible.

The following guide will get you up and running and developing/debugging CHEFS as quickly as possible.
We provide a [`devcontainer`](https://containers.dev) and will use [`VS Code`](https://code.visualstudio.com) to illustrate.

By no means is CHEFS development limited to these tools; they are merely examples.
By no means is CHEFS development limited to these tools; they are merely examples.

## Caveats

Expand All @@ -11,6 +12,7 @@ The primary use case for this `devcontainer` is for developing, debugging and un
There are limitations running this devcontainer, such as all networking is within this container. This container has [docker-in-docker](https://github.com/microsoft/vscode-dev-containers/blob/main/script-library/docs/docker-in-docker.md) which allows running demos, building docker images, running `docker compose` all within this container.

## Files

The `.devcontainer` folder contains the `devcontainer.json` file which defines this container. We are using a `Dockerfile` and `post-install.sh` to build and configure the container run image. The `Dockerfile` is simple but in place for simplifying image enhancements. The `post-install.sh` will install the required node libraries for CHEFS including the frontend and formio components.

In order to run CHEFS you require Keycloak (configured), Postgresql (seeded) and the CHEFS backend/API and frontend/UX. Previously, this was a series of downloads and configuration updates and numerous commands to run. See `.devcontainer/chefs_local` files.
Expand All @@ -21,42 +23,46 @@ Also included are convenient launch tasks to run and debug CHEFS.

## Open CHEFS in the devcontainer

To open CHEFS in a devcontainer, we open the *root* of this repository. We can open in 2 ways:
To open CHEFS in a devcontainer, we open the _root_ of this repository. We can open in 2 ways:

1. Open Visual Studio Code, and use the Command Palette and use `Dev Containers: Open Folder in Container...`
2. Open Visual Studio Code and `File|Open Folder...`, you should be prompted to `Reopen in Container`.


## Running CHEFS locally

Keycloak and Postgresql will be launched using docker compose. These will run inside of the devcontainer (docker-in-docker) but the ports are forwarded to the host machine and are accessible on the local host.

CHEFS API and Frontend are running as node applications on the devcontainer - again, ports are forwarded to the host.

### Configuring CHEFS locally
When the devcontainer is built, it copies `.devcontainer/chefs_local/local.json.sample` and `.devcontainer/chefs_local/realm-export.json.sample` to `.devcontainer/chefs_local/local.json` and `.devcontainer/chefs_local/realm-export.json` respectively. These copies are not checked in and allow the developer to make changes and tweaks without impacting other developers or accidentially committing passwords.

When the devcontainer is built, it copies `.devcontainer/chefs_local/local.json.sample` and `.devcontainer/chefs_local/realm-export.json.sample` to `.devcontainer/chefs_local/local.json` and `.devcontainer/chefs_local/realm-export.json` respectively. These copies are not checked in and allow the developer to make changes and tweaks without impacting other developers or accidentially committing passwords.

### Authorization Prerequisites
1. An IDIR account is required to access CHEFS.
2. Request an SSO Integration from the Common Hosted Single Sign-on (CSS) page in order to obtain a resource and secret that will be used for authentication when building CHEFS. View the [detailed documentation](https://bcdevex.atlassian.net/wiki/spaces/CCP/pages/961675282) about requesting the Pathfinder SSO integration.
3. Open realm-export.json located at chefs_build/docker/imports/keycloak and search for `XXXXXXXXXXXX`. This value must match the `clientSecret` value in `local.json` so that the CHEFS API can connect to your Keycloak instance. By default, these are set to be equal and don’t need to be altered.
4. Navigate to the CSS page, login with your IDIR, and download the ‘Development’ Installation JSON from your SSO Integration.
5. Back in the `realm-export.json` file, search for all instances of `YYYYYYYYYYYY` and replace it with the `resource` you obtained from the downloaded JSON file. Search for all instances of `ZZZZZZZZZZZZ` and replace it with the `secret`.

1. An IDIR account is required to access CHEFS.
2. Request an SSO Integration from the Common Hosted Single Sign-on (CSS) page in order to obtain a resource and secret that will be used for authentication when building CHEFS. View the [detailed documentation](https://bcdevex.atlassian.net/wiki/spaces/CCP/pages/961675282) about requesting the Pathfinder SSO integration.
3. Open realm-export.json located at build/docker/imports/keycloak and search for `XXXXXXXXXXXX`. This value must match the `clientSecret` value in `local.json` so that the CHEFS API can connect to your Keycloak instance. By default, these are set to be equal and don’t need to be altered.
4. Navigate to the CSS page, login with your IDIR, and download the ‘Development’ Installation JSON from your SSO Integration.
5. Back in the `realm-export.json` file, search for all instances of `YYYYYYYYYYYY` and replace it with the `resource` you obtained from the downloaded JSON file. Search for all instances of `ZZZZZZZZZZZZ` and replace it with the `secret`.

### Run/Debug
1. start Keycloak and Postgresql. Many ways to start...
- right click on `.devcontainer/chefs_local/docker-compose.yml` and select `Compose up`
- or use command palette `Docker: Compose Up` then select `.devcontainer/chefs_local/docker-compose.yml`
- or `Terminal | Run Task...|chefs_local up`

1. start Keycloak and Postgresql. Many ways to start...
- right click on `.devcontainer/chefs_local/docker-compose.yml` and select `Compose up`
- or use command palette `Docker: Compose Up` then select `.devcontainer/chefs_local/docker-compose.yml`
- or `Terminal | Run Task...|chefs_local up`
2. start CHEFS
- Run and Debug, select 'CHEFS' which will start both the API and the frontend.
- Run and Debug, select 'CHEFS' which will start both the API and the frontend.
3. debug Frontend with Chrome
- Run and Debug, select 'CHEFS Frontend - chrome' which will start a Chrome browser against the frontend, will allow breakpoints in `/app/frontend/src`
4. stop Keycloak and Postgresql. Many ways to stop...
- right click on `.devcontainer/chefs_local/docker-compose.yml` and select `Compose down`
- or use command palette `Docker: Compose Down` then select `.devcontainer/chefs_local/docker-compose.yml`
- or `Terminal | Run Task...|chefs_local down`
- Run and Debug, select 'CHEFS Frontend - chrome' which will start a Chrome browser against the frontend, will allow breakpoints in `/app/frontend/src`
4. stop Keycloak and Postgresql. Many ways to stop...
- right click on `.devcontainer/chefs_local/docker-compose.yml` and select `Compose down`
- or use command palette `Docker: Compose Down` then select `.devcontainer/chefs_local/docker-compose.yml`
- or `Terminal | Run Task...|chefs_local down`

_Notes_

*Notes*
- `CHEFS Frontend` launch configuration is using the `chefs-frontend-local` client in Keycloak, not `chefs-frontend` client as we do in production.
- `CHEFS API` will use the configuration found at `.devcontainer/chefs_local/local.json`
- `Postgres DB`: localhost:5432
Expand All @@ -65,6 +71,7 @@ When the devcontainer is built, it copies `.devcontainer/chefs_local/local.json.
- `CHEFS API`: http://localhost:5173/app/api/v1

## Formio Components

If you are developing the formio components, you should build and redeploy them before running your local debug instances of CHEFS. Use tasks `Components build` and `Components Deploy`.

## KNEX - Database tools
Expand Down Expand Up @@ -99,9 +106,11 @@ knex migrate:rollback
Please review the [knex](https://knexjs.org) for more detail and how to leverage the tool.

## Troubleshooting

All development machines are unique and here we will document problems that have been encountered and how to fix them.

### Failure during load of devcontainer when running webpack (Segmentation Fault)

Encountered on Mac Ventura 13.6, with Mac Docker Desktop 4.26.1 when running `npm run build:formio` on load, we hit a `Segmentation Fault`. The issue was resolved when turning off the virtualization settings in Docker Desktop.

Under Settings, select `gRPC Fuse` instead of `VirtioFS` then unselect `Use Virtualization framework`. Restart Docker and VS Code.
Under Settings, select `gRPC Fuse` instead of `VirtioFS` then unselect `Use Virtualization framework`. Restart Docker and VS Code.
93 changes: 45 additions & 48 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -1,58 +1,55 @@
// For format details, see https://aka.ms/devcontainer.json. For config options, see the
{
"name": "common-hosted-form-service",

"build": {
"dockerfile": "Dockerfile",
"context": "..",
"args": {
"VARIANT": "18.18.2-bullseye"
}
},
"name": "common-hosted-form-service",

"customizations": {
"vscode": {
"extensions": [
"cweijan.vscode-postgresql-client2",
"Vue.volar",
"esbenp.prettier-vscode"
],
"settings": {
"database-client.telemetry.usesOnlineServices": false,
"editor.defaultFormatter": null,
"editor.formatOnSave": false,
"[javascript]": {
"editor.defaultFormatter": "esbenp.prettier-vscode",
"editor.formatOnSave": true
},
"prettier.configPath": "${containerWorkspaceFolder}/app/frontend/.prettierrc",
"prettier.documentSelectors": ["${containerWorkspaceFolder}/app/frontend/**/*.{js,vue}"]
}
}
},
"build": {
"dockerfile": "Dockerfile",
"context": "..",
"args": {
"VARIANT": "18.18.2-bullseye"
}
},

"features": {
"ghcr.io/devcontainers/features/docker-in-docker:2": {}
},
"features": {
"ghcr.io/devcontainers/features/docker-in-docker:2": {}
},

"containerEnv": {
"NODE_CONFIG_DIR": "${containerWorkspaceFolder}/.devcontainer/chefs_local"
},

// Use this environment variable if you need to bind mount your local source code into a new container.
"remoteEnv": {
"LOCAL_WORKSPACE_FOLDER": "${localWorkspaceFolder}"
},
// Use this environment variable if you need to bind mount your local source code into a new container.
"remoteEnv": {
"LOCAL_WORKSPACE_FOLDER": "${localWorkspaceFolder}"
},

// Use 'forwardPorts' to make a list of ports inside the container available locally.
"forwardPorts": [8082, 8081, 8080, 5432, 5173],
// Use 'forwardPorts' to make a list of ports inside the container available locally.
"forwardPorts": [
5173, // CHEFS Frontend
5432, // PostgreSQL
8080, // CHEFS Backend
8081,
8082 // Keycloak
],

// Use 'postCreateCommand' to run commands after the container is created.
"postCreateCommand": "bash ./.devcontainer/post-install.sh",
// Use 'postCreateCommand' to run commands after the container is created.
"postCreateCommand": "bash ./.devcontainer/post-install.sh",

// Configure tool-specific properties.
// "customizations": {},
// Configure tool-specific properties.
"customizations": {
"vscode": {
"extensions": [
"cweijan.vscode-postgresql-client2", // PostgreSQL client
"dbaeumer.vscode-eslint", // ESLint to catch problems early
"esbenp.prettier-vscode", // Prettier to format files on save
"postman.postman-for-vscode", // Postman for integration tests
"redocly.openapi-vs-code", // ReDocly to catch OpenAPI errors
"vue.volar" // Vue 3 recommended extension
],
"settings": {
"database-client.telemetry.usesOnlineServices": false,
"editor.defaultFormatter": "esbenp.prettier-vscode",
"editor.formatOnSave": true
}
}
}

// Uncomment to connect as root instead. More info: https://aka.ms/dev-containers-non-root.
//"remoteUser": "root"
// Uncomment to connect as root instead. More info: https://aka.ms/dev-containers-non-root.
//"remoteUser": "root"
}
10 changes: 5 additions & 5 deletions .github/actions/build-push-container/action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -69,15 +69,16 @@ runs:
using: composite
steps:
- name: Checkout repository from pull request
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
ref: ${{ inputs.ref }}
if: ${{ inputs.ref != '' }}
- name: Checkout repository
uses: actions/checkout@v3
uses: actions/checkout@v4
if: ${{ inputs.ref == '' }}

- name: Set variables
shell: bash
run: |
echo "SHA=sha-$(git rev-parse --short HEAD)" >> $GITHUB_ENV
echo "IMAGE_REVISION=$(git rev-parse HEAD)" >> $GITHUB_ENV
Expand Down Expand Up @@ -148,9 +149,8 @@ runs:
latest=true
# Creates tags based off of branch names and semver tags
tags: |
type=raw,value=ghcr.io/${{ env.GH_USERNAME }}/${{ inputs.image_name }}:${{ env.IMAGE_VERSION }}
type=raw,value=ghcr.io/${{ env.GH_USERNAME }}/${{ inputs.image_name }}:${{ env.SHA }}
type=raw,value=ghcr.io/${{ env.GH_USERNAME }}/${{ inputs.image_name }}:latest
type=raw,value=${{ env.IMAGE_VERSION }}
type=raw,value=${{ env.SHA }}
labels: |
org.opencontainers.image.revision=${{ env.IMAGE_REVISION }}
org.opencontainers.image.version=${{ env.IMAGE_VERSION }}
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/.close-pr.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ jobs:
timeout-minutes: 12
steps:
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: Login to OpenShift Cluster
uses: redhat-actions/oc-login@v1
with:
Expand All @@ -48,4 +48,4 @@ jobs:
with:
header: release
delete: true
number: ${{ github.event.inputs.pr-number }}
number: ${{ github.event.inputs.pr-number }}
37 changes: 12 additions & 25 deletions .github/workflows/.deploy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,18 +7,9 @@ on:
workflow_dispatch:
inputs:
pr-number:
description: Pull request number, leave blank for dev/test/prod deployment
description: Pull request number
required: false
type: string
environment:
description: Environment name; choose dev for PR
required: true
type: choice
options:
- dev
- test
- prod
default: dev

concurrency:
group: ${{ github.workflow }}-${{ github.event.inputs.pr-number || github.ref }}
Expand All @@ -30,7 +21,6 @@ jobs:
runs-on: ubuntu-latest
outputs:
APP_TITLE: ${{ steps.vars.outputs.APP_TITLE }}
ENVIRONMENT: ${{ steps.vars.outputs.ENVIRONMENT }}
JOB_NAME: ${{ steps.vars.outputs.JOB_NAME }}
ROUTE_PATH: ${{ steps.vars.outputs.ROUTE_PATH }}
URL: ${{ steps.vars.outputs.URL }}
Expand All @@ -40,14 +30,12 @@ jobs:
id: default-vars
env:
PR_NUMBER: ${{ github.event.inputs.pr-number }}
ENVIRONMENT: ${{ github.event.inputs.environment }}
ACRONYM: ${{ env.ACRONYM }}
run: |
echo "APP_TITLE=Common Hosted Forms" >> "$GITHUB_OUTPUT"
echo "ENVIRONMENT=$ENVIRONMENT" >> "$GITHUB_OUTPUT"
echo "JOB_NAME=master" >> "$GITHUB_OUTPUT"
echo "ROUTE_PATH=/app" >> "$GITHUB_OUTPUT"
echo "URL=https://$ACRONYM-$ENVIRONMENT.apps.silver.devops.gov.bc.ca" >> "$GITHUB_OUTPUT"
echo "URL=https://$ACRONYM-dev.apps.silver.devops.gov.bc.ca" >> "$GITHUB_OUTPUT"
- name: Final variables
id: vars
env:
Expand All @@ -58,13 +46,11 @@ jobs:
echo "ref=$REF" >> $GITHUB_OUTPUT
if [[ "$PR_NUMBER" != '' ]]; then
echo "APP_TITLE=${{ steps.default-vars.outputs.APP_TITLE }} - PR-$PR_NUMBER" >> "$GITHUB_OUTPUT"
echo "ENVIRONMENT=pr" >> "$GITHUB_OUTPUT"
echo "JOB_NAME=pr-$PR_NUMBER" >> "$GITHUB_OUTPUT"
echo "ROUTE_PATH=/pr-$PR_NUMBER" >> "$GITHUB_OUTPUT"
echo "URL=${{ steps.default-vars.outputs.URL }}/pr-$PR_NUMBER" >> "$GITHUB_OUTPUT"
else
echo "APP_TITLE=${{ steps.default-vars.outputs.APP_TITLE }}" >> "$GITHUB_OUTPUT"
echo "ENVIRONMENT=${{ steps.default-vars.outputs.ENVIRONMENT }}" >> "$GITHUB_OUTPUT"
echo "JOB_NAME=${{ steps.default-vars.outputs.JOB_NAME }}" >> "$GITHUB_OUTPUT"
echo "ROUTE_PATH=${{ steps.default-vars.outputs.ROUTE_PATH }}" >> "$GITHUB_OUTPUT"
echo "URL=${{ steps.default-vars.outputs.URL }}/app" >> "$GITHUB_OUTPUT"
Expand All @@ -77,12 +63,12 @@ jobs:
timeout-minutes: 10
steps:
- name: Checkout repository from pull request
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
ref: ${{ needs.set-vars.outputs.ref }}
if: ${{ needs.set-vars.outputs.ref != '' }}
- name: Checkout repository
uses: actions/checkout@v3
uses: actions/checkout@v4
if: ${{ needs.set-vars.outputs.ref == '' }}
- name: Build & Push
uses: ./.github/actions/build-push-container
Expand All @@ -100,29 +86,29 @@ jobs:
deploy:
name: Deploys to selected environment
environment:
name: ${{ needs.set-vars.outputs.ENVIRONMENT }}
name: pr
url: ${{ needs.set-vars.outputs.URL }}
runs-on: ubuntu-latest
needs: [set-vars, build]
timeout-minutes: 12
steps:
- name: Checkout repository from pull request
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
ref: ${{ needs.set-vars.outputs.ref }}
if: ${{ needs.set-vars.outputs.ref != '' }}
- name: Checkout repository
uses: actions/checkout@v3
uses: actions/checkout@v4
if: ${{ needs.set-vars.outputs.ref == '' }}
- name: Deploy to environment
uses: ./.github/actions/deploy-to-environment
with:
app_name: ${{ vars.APP_NAME }}
acronym: ${{ env.ACRONYM }}
environment: ${{ needs.set-vars.outputs.ENVIRONMENT }}
environment: pr
job_name: ${{ needs.set-vars.outputs.JOB_NAME }}
namespace_prefix: ${{ vars.NAMESPACE_PREFIX }}
namespace_environment: ${{ github.event.inputs.environment }}
namespace_environment: dev
openshift_server: ${{ secrets.OPENSHIFT_SERVER }}
openshift_token: ${{ secrets.OPENSHIFT_TOKEN }}
server_host: ${{ vars.SERVER_HOST }}
Expand All @@ -131,8 +117,9 @@ jobs:
ref: ${{ needs.set-vars.outputs.ref }}
- name: Release Comment on PR
uses: marocchino/sticky-pull-request-comment@v2
if: github.event.inputs.pr-number != '' && success()
if: ${{ github.event.inputs.pr-number }} != '' && success()
with:
header: release
message: |
Release ${{ github.sha }} deployed at <https://${{ env.ACRONYM }}-dev.apps.silver.devops.gov.bc.ca/pr-${{ github.event.inputs.pr-number }}>
Release ${{ github.sha }} deployed at <https://${{ env.ACRONYM }}-dev.apps.silver.devops.gov.bc.ca/pr-${{ github.event.inputs.pr-number }}>
number: ${{ github.event.inputs.pr-number }}
Loading

0 comments on commit 7c610cd

Please sign in to comment.