Skip to content

omaripaul57/rubin-rucio-deploy2

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

rubin-rucio-deploy

Deployment framework for Rubin USDF Rucio.

Once you have access to your Kubernetes cluster, you can deploy Rucio for a given overlay (found in overlays/[dev,prod]) using the Makefile found there with (most simply)

make apply

This will run multiple steps, downloading Secrets from Vault and ending up using Kustomize to create or update rucio and rucio-db namespaces and the resources to run a Rucio application in those namespaces. Rucio application containers will run in the rucio namespace, while the rucio-db namespace is used for a CloudNativePG operator managed PostgreSQL database.

This framework uses kustomize to allow modification of Helm template outputs from the official Rucio Helm chart. This is used in particular to patch the environment of containers such that they are configured to have the database connection string passed in via the value of a Secret. This allows us to keep the Kubernetes resource manifests generated by the Helm templates under version control, without exposing a base64 encoded password as compared with a --values injection to the Helm template process.

Before deploying Rucio, the secrets and credentials needed for the application will need to be pushed to Vault with the secrets2vault.sh script. Vault-resident secrets may then be downloaded when make apply is run, and then loaded into the cluster via SecretGenerator definitions in the kustomization.yaml files. The format of the secret directory that the secrets2vault.sh script uses should be as follows:

secret
├── ca.pem
├── dev
│   ├── hostcert.pem
│   └── hostkey.pem
├── pg-conn-str.txt
└── prod
    ├── hostcert.pem
    └── hostkey.pem

If you update the values-*.yaml files, be sure to also update the rucio helm templates with:

make rucio  # update from upstream helm charts
make apply

Be very careful with the make destroy command. It will delete everything, including your database PersistentVolume objects.

About

Rucio deployment framework for the Rubin Observatory

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 75.1%
  • Makefile 13.4%
  • Shell 9.8%
  • Dockerfile 1.7%