Demo Idea: Exemplary GitOps Setup with Stackable

[stackfab]

Stackable data platforms are defined declaratively, enabling automated, standardized, and efficient deployments.

This demo showcases an exemplary GitOps setup for the Stackable data platform using e.g. ArgoCD and distinct Dev/Prod stages. This approach offers consistency, version control, speed, enhanced collaboration, robust security, and observability, making it easier for teams to manage infrastructure as code and adopt Stackable.

For new, non-advanced users, this setup simplifies the learning curve, reduces manual errors, and provides a clear pathway to modern DevOps/GitOps practices.

Prerequisites

• no Dev/Prod stages (only 1 cluster)
• without terraform
• including secrets via git (sealed secrets)

Tasks - GitOps

• Provide Argo Stack (Argo, Sealed-Secrets, Stackable-operators)
• Argo-fy stackable operators, airflow and its dependencies
• Show ArgoCD UI
• Provide guide to scale settings via gitops (e.g. scale webserver replicas) or add new DAGs via git-sync
• Show Airflow UI and start DAGs and view logs
• Documentation

[stefanigel]

There is already some ArgoCD experience from some projects that can serve as an input.

[dervoeti]

When doing this, I think it would be good to at least showcase how secret management can be done. Like: Don't store plaintext secrets in Git, encrypt them or fetch them from a store like Vault or AWS Secrets Manager. Let ArgoCD or some Kubernetes controller do the decryption/fetching.

[soenkeliebau]

Example of how this might possible look here: https://github.com/stackabletech/demos/tree/spike/argocd-demo

[stefanigel]

Talking to some CSVs and integration partners recently: They often start with some Terraform scripts. Can we consider this (in this issue or as a phase 2)?

[stackfab]

Ran across this one. Might me worth checking out:

https://github.com/zapier/kubechecks