K3D

Deploy persistence kubernetes cluster

Crete a directory in your host where Kubernetes cluster will be persist data

> mkdir ./k3dvol

Create Kubernetes Cluster with LoadBalancer

NOTE: Master and Workers nodes are renamed to Server and Agents resp.

Create a Kubernetes Cluster. For this sample will keep cluster simple but you can set "any number" of agents and workers, the limits is comon sense and your memory.

Note that we are pointing port 443 on host to Cluster Load Balancer's 443 port. If you want use http you can use port 80.

> k3d cluster create dev-cluster \
--api-port 6443 \
--port 8443:443@loadbalancer  \
--port 8080:80@loadbalancer \
--volume $(pwd)/k3dvol:/tmp/k3dvol \
--servers 1 --agents 1

Port Mapping

• --port 8080:80@loadbalancer will add a mapping of local host port 8080 to loadbalancer port 80, which will proxy requests to port 80 on all agent nodes

• --api-port 6443 : by default, no API-Port is exposed (no host port mapping). It’s used to have k3s‘s API-Server listening on port 6553 with that port mapped to the host system. So that the load balancer will be the access point to the Kubernetes API, so even for multi-server clusters, you only need to expose a single api port. The load balancer will then take care of proxying your requests to the appropriate server node

• --port "32000-32767:32000-32767@loadbalancer" You may as well expose a NodePort range (if you want to avoid the Ingress Controller). Warning: Map a wide range of ports can take a certain amount of time, and your computer can freeze for some time in this process.

Manage Clusters

Once cluster is created we can start, stop or even delete them

> k3d cluster start <cluster-name>
> k3d cluster stop <cluster-name>
> k3d cluster delete <cluster-name>`

Mange cluser nodes

List cluster nodes

> k3d node ls
NAME                       ROLE           CLUSTER       STATUS
k3d-k3d-cluster-agent-0    agent          k3d-cluster   running
k3d-k3d-cluster-agent-1    agent          k3d-cluster   running
k3d-k3d-cluster-server-0   server         k3d-cluster   running
k3d-k3d-cluster-serverlb   loadbalancer   k3d-cluster   running

Add/Delete new nodes to cluster

Create new nodes (and add them to existing clusters)

> k3d node create <nodename> --cluster <cluster-name> --role <agent/server>

To delete nodes just use:

> k3d node delete <nodename>

Example

> k3d node ls
NAME                       ROLE           CLUSTER       STATUS
k3d-k3d-cluster-agent-0    agent          k3d-cluster   running
k3d-k3d-cluster-agent-1    agent          k3d-cluster   running
k3d-k3d-cluster-server-0   server         k3d-cluster   running
k3d-k3d-cluster-serverlb   loadbalancer   k3d-cluster   running

> k3d node create my-new-agent-node --cluster k3d-cluster --role agent
INFO[0000] Starting Node 'k3d-my-new-agent-node-0'

> k3d node ls
NAME                       ROLE           CLUSTER       STATUS
k3d-k3d-cluster-agent-0    agent          k3d-cluster   running
k3d-k3d-cluster-agent-1    agent          k3d-cluster   running
k3d-k3d-cluster-server-0   server         k3d-cluster   running
k3d-k3d-cluster-serverlb   loadbalancer   k3d-cluster   running
k3d-my-new-agent-node-0    agent          k3d-cluster   running

> k3d node delete  k3d-my-new-agent-node-0
INFO[0000] Deleted k3d-my-new-agent-node-0   

> k3d node ls
NAME                       ROLE           CLUSTER       STATUS
k3d-k3d-cluster-agent-0    agent          k3d-cluster   running
k3d-k3d-cluster-agent-1    agent          k3d-cluster   running
k3d-k3d-cluster-server-0   server         k3d-cluster   running
k3d-k3d-cluster-serverlb   loadbalancer   k3d-cluster   running

Start/Stop nodes

Also can just stop or start nodes previously created with

> k3d node start <nodename>
> k3d node stop <nodename>

k3d create/start/stop/delete node mynode

Manage your registry

Create or delete a local kubernetes internal registry

> k3d registry create REGISTRY_NAME 
> k3d registry delete REGISTRY_NAME

Replace Ingress Controler

K3D uses Traefik 1.x versios as Ingress controler, due Traefik 2.x is enough mature and provide more functionalities we need do some extra work to use Traefik.

First we create a new file helm-ingress-traefik.yaml

# see https://rancher.com/docs/k3s/latest/en/helm/
# see https://github.com/traefik/traefik-helm-chart
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: ingress-controller-traefik
  namespace: kube-system
spec:
  repo: https://helm.traefik.io/traefik
  chart: traefik
  version: 9.8.0
  targetNamespace: kube-system

Now we can create a new cluster telling to k3d not deploy traefik with --k3s-server-arg '--no-deploy=traefik' and use previous helm chart defined to deploy new Traefik Ingress Controler --volume "$(pwd)/helm-ingress-traefik.yaml:/var/lib/rancher/k3s/server/manifests/helm-ingress-traefik.yaml"

> k3d cluster create traefik --k3s-server-arg '--no-deploy=traefik' --volume "$(pwd)/helm-ingress-traefik.yaml:/var/lib/rancher/k3s/server/manifests/helm-ingress-traefik.yaml"