Flux2 Infra repository

This repository contains Infrastructure applications and addons installed inside Kubernetes.

Table of Contents

• Support Project
• Prerequisites
• List of applications
• Import current repository
  • Substitute variables
• Repository structure
• Structure idea
• Flex/Stable Bundles and promotion
• Slack Notifications

Support Project

Prerequisites

• Kubernetes cluster version 1.24 or newer
• Flux version 2.3.0 or newer bootstrapped to the Head repository (example)
• CRD GitOps repository must be included before this as a dependency.

List of applications

• Apache Airflow
• ArgoCD
• aws-load-balancer-controller
• blackbox-exporter
• cert-manager
• cloudnative-pg (PostgreSQL operator)
• cnpg tenant example
• pgadmin
• Dragonfly Operator (redis replacement)
• Dragonfly Instance (redis replacement)
• Clickhouse Operator (Altinity)
• Clickhouse simple example (Altinity)
• cluster-autoscaler
• external-dns
• external-secrets
• flux-monitoring (alerts and dashboards for FluxCD)
• helm-exporter
• Istio ServiceMesh
• Jenkins
• k8s-event-logger
• KEDA
• kubelinks
• metrics-server
• reflector
• Strimzi Kafka Operator
• Victoria Metrics (victoria-metrics-k8s-stack)
• Loki
• Grafana Alloy
• capacitor (Flux UI)
• ollama & open-webui
• minio-operator
• minio-tenant example
• SeaweedFS (S3 alternative)
• elastic operator (ECK)
• oomkill-exporter
• x509-certificate-exporter
• stakater/Reloader

Import current repository

Create GitRepository and Flux Kustomization in your main repository (change path and substitute variable for related):

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: fluxcd-gitops-infra
  namespace: flux-system
spec:
  interval: 10m
  ref:
    branch: main
  secretRef:
    name: flux-system
  url: https://github.com/brainfair/awesome-flux-infra.git
  ignore: |
 # exclude README.md
 /README.md
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: fluxcd-gitops-infra
  namespace: flux-system
spec:
  dependsOn:
 - name: fluxcd-gitops-crds
  interval: 10m
  path: ./clusters/homelab
  prune: true
  sourceRef:
    kind: GitRepository
    name: fluxcd-gitops-infra
  postBuild:
    substitute:
      env: "homelab"
      cluster_name: "docker-desktop"
      cluster_subdomain: "localhost.direct"

Substitute variables

• env - environment name
• cluster_name - the name of the Kubernetes cluster
• cluster_subdomain - subdomain for all ingress resources

Check head repository example

Repository structure

The Git repository contains the following top directories:

• apps/base dir contains base application objects such as helmrelease, helmrepository, namespace, etc...
• apps/bundles dir contains bundles for different types of environment
• clusters dir contains cluster entry point where we can include different bundles or custom apps

── apps
 ├── base
 ├   ├── aws-load-balancer-controller
 ├   ├── blackbox-exporter
 ├   ....
 ├── bundles
    ├── eks-flex
    ├── eks-stable
    ├── aks-flex
    ├── aks-stable
    ├── esxi-flex
    ├── esxi-stable
    ├── docker-flex
    ├── docker-stable
    ├── ...
── clusters
    ├── eks-cluster
    ├── aks-cluster
    ├── dockerdesktop-cluster
    ├── vmware-cluster
    ├── ...

Structure idea

The basic idea to define 3 levels of application kustomization:

• base level inside apps/base defines a common definition of application and values for everything
• bundles level inside apps/bundles defines aggregation of the base application and kustomization for the specified bundle
• cluster level inside clusters/[cluster_name] defines entry point where we can include some bundle and override cluster-specific values or add something more

Flex/Stable Bundles and promotion

To keep infrastructure up-to-date we defined two bundles for the Docker Desktop environment.

• docker-flex - where we define the version range for every application (block major updates) for automatic updates
• docker-stable - where we define a pinned version for each application.

Non-production environment should include a flex bundle where we can play/test/evaluate new applications and new versions. Production environments should be a pointer to a stable bundle.

For 3rd party applications when a new version is successfully updated in the flex bundle we run the promotion workflow triggered by dispatch notification defined in the staging docker cluster.