Skip to content

Latest commit

 

History

History
103 lines (72 loc) · 3.68 KB

manage-infrastructure.md

File metadata and controls

103 lines (72 loc) · 3.68 KB

Manage Infrastructure

The service is hosted on the MoJ Cloud Platform. It's a platform where we can host our applications and interact with them through Kubernetes. This requires our applications to all be Dockerised.

Each repository will have a helm_deploy directory that specifies configuration for this service in each of the environments.

When this app deploys via merging to main, CircleCI will automatically propagate those changes to the cluster. This happens through the hmpps/deploy_env job that is provided by the MoJ CircleCI Orb.

Each environment will correspond to a Cloud Platform 'namespace'. The namespace is an isolated cluster. We can use the Cloud Platform Environments repository to define our backing services, certificates etc.

Prerequisites

Kubernetes cheat sheet

To use Kubernetes to interact with the cluster there's a cheat sheet. Keep reading for tasks we commonly use.

Common Kubernetes tasks

View the application logs of a pod

Find the name of the pod you'd like to get the logs for:

kubectl --namespace hmpps-community-accommodation-<env> get pods

Follow the logs:

kubectl -n hmpps-community-accommodation-<env> logs --follow <pod name> --all-containers

View/change the value of an environment variable

Environment variables are themselves defined with Helm.

For environment variables that aren't secrets we can set these values in our Helm charts.

For environment variables that contain secrets we can't set these in GitHub so we have to set the values by hand.

First find the secret set you'd like to view/change:

kubectl --namespace hmpps-community-accommodation-<env> get secrets

Add the secret name and view or make the change:

kubectl --namespace hmpps-community-accommodation-<env> edit secret <secret set name>

Consider a rolling restart to apply this change.

Rolling restart

Restart an individual service without downtime. Each service will have multiple containers running. This process will attempt to start a new replica set alongside the existing set that's currently serving real requests. If the new set is healthy, Kubernetes will gracefully replace the existing set and then remove the old. Useful as one way to refresh environment variables.

First find the service name you'd like to restart:

kubectl --namespace hmpps-community-accommodation-<env> get services

Start the restart:

kubectl --namespace hmpps-community-accommodation-<env> rollout restart deployment <service name>

You can observe the progress if you like:

watch kubectl --namespace hmpps-community-accommodation-<env> get pods