Creates a Single Controller Kubernetes cluster using multipass and cilium
All scripts are available to learn how it is built.
- Ubuntu 22.04
- Kubernetes 1.28.4
- etcd 3.5.10
- containerd 1.7.10
- cni plugins 1.4.0
- cilium 1.14.4 (via helm chart)
- coredns 1.28.2 (via helm chart)
- helm 3+
- linux: here
- mac:
brew install helm
- curl
- linux:
sudo apt install curl - mac:
brew install curl
- linux:
- ipcalc
- linux:
sudo apt install ipcalc - mac:
brew install ipcalc
- linux:
- multipass
- linux:
sudo snap install multipass --classic - mac:
brew cask install multipass
- linux:
cfsslandcfssljson-
linux:
wget -q --show-progress --https-only --timestamping \ https://storage.googleapis.com/kubernetes-the-hard-way/cfssl/linux/cfssl \ https://storage.googleapis.com/kubernetes-the-hard-way/cfssl/linux/cfssljson chmod +x cfssl cfssljson sudo mv cfssl cfssljson /usr/local/bin/
-
mac:
brew install cfssl
-
kubectl-
linux:
wget https://storage.googleapis.com/kubernetes-release/release/v1.18.5/bin/linux/amd64/kubectl chmod +x kubectl sudo mv kubectl /usr/local/bin/
-
mac:
curl -o kubectl https://storage.googleapis.com/kubernetes-release/release/v1.18.5/bin/darwin/amd64/kubectl chmod +x kubectl sudo mv kubectl /usr/local/bin/
-
-
Create your machines:
./setup.sh
-
Wait a couple of minutes for cilium and coredns to start working
$ kubectl get all --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE kube-system pod/cilium-25lnk 1/1 Running 0 73s kube-system pod/cilium-d8p59 1/1 Running 0 74s kube-system pod/cilium-operator-67f9b484d8-ksvzj 1/1 Running 0 74s kube-system pod/cilium-operator-67f9b484d8-mqvsq 1/1 Running 0 74s kube-system pod/coredns-coredns-67c645769c-klwlk 1/1 Running 0 73s kube-system pod/hubble-relay-85664d47c4-ftd5n 1/1 Running 0 74s kube-system pod/hubble-ui-7c6789876c-7b9mh 3/3 Running 0 74s NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE default service/kubernetes ClusterIP 172.17.0.1 <none> 443/TCP 2m31s kube-system service/coredns-coredns ClusterIP 172.17.0.10 <none> 53/UDP,53/TCP 73s kube-system service/hubble-relay ClusterIP 172.17.0.83 <none> 80/TCP 74s kube-system service/hubble-ui ClusterIP 172.17.0.35 <none> 80/TCP 74s NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE kube-system daemonset.apps/cilium 2 2 2 2 2 <none> 74s NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE kube-system deployment.apps/cilium-operator 2/2 2 2 74s kube-system deployment.apps/coredns-coredns 1/1 1 1 73s kube-system deployment.apps/hubble-relay 1/1 1 1 74s kube-system deployment.apps/hubble-ui 1/1 1 1 74s NAMESPACE NAME DESIRED CURRENT READY AGE kube-system replicaset.apps/cilium-operator-67f9b484d8 2 2 2 74s kube-system replicaset.apps/coredns-coredns-67c645769c 1 1 1 73s kube-system replicaset.apps/hubble-relay-85664d47c4 1 1 1 74s kube-system replicaset.apps/hubble-ui-7c6789876c 1 1 1 74s
-
Your cluster is ready, lets verify data encryption works
$ kubectl create secret \ generic kubernetes-the-hard-way --from-literal="mykey=mydata" $ multipass exec controller-k8s -- sudo ETCDCTL_API=3 etcdctl get \ --endpoints=https://127.0.0.1:2379 --cacert=/etc/etcd/ca.pem \ --cert=/etc/etcd/kubernetes.pem --key=/etc/etcd/kubernetes-key.pem \ /registry/secrets/default/kubernetes-the-hard-way | hexdump -C 00000000 2f 72 65 67 69 73 74 72 79 2f 73 65 63 72 65 74 |/registry/secret| 00000010 73 2f 64 65 66 61 75 6c 74 2f 6b 75 62 65 72 6e |s/default/kubern| 00000020 65 74 65 73 2d 74 68 65 2d 68 61 72 64 2d 77 61 |etes-the-hard-wa| 00000030 79 0a 6b 38 73 3a 65 6e 63 3a 61 65 73 63 62 63 |y.k8s:enc:aescbc| 00000040 3a 76 31 3a 6b 65 79 31 3a 61 bb c0 45 f2 df 88 |:v1:key1:a..E...| 00000050 36 46 05 df c1 df 26 e1 e0 59 18 9f 7d 51 7a d9 |6F....&..Y..}Qz.| 00000060 28 0d 03 4e c3 14 55 01 51 d6 aa cc 50 21 a5 09 |(..N..U.Q...P!..| 00000070 86 92 89 9b 33 82 43 09 7d 5b fe bb 68 45 43 48 |....3.C.}[..hECH| 00000080 96 9a 1e a8 88 30 82 a8 2c d8 26 ea 12 19 58 da |.....0..,.&...X.| 00000090 3a 25 ed 6b 47 1f e2 e9 31 91 e6 cf 64 bb 19 41 |:%.kG...1...d..A| 000000a0 fe 2b 7a 86 a8 be e4 c0 b6 98 2e dc 96 92 58 92 |.+z...........X.| 000000b0 c4 6b c1 85 a9 d0 ec d6 03 2d c7 2c 14 f5 da 03 |.k.......-.,....| 000000c0 ef c6 c9 2b bc 26 9c 36 ab 0c da 08 f2 8b 79 c7 |...+.&.6......y.| 000000d0 12 98 55 5f 4c 56 f7 fd e1 71 45 16 a3 59 01 76 |..U_LV...qE..Y.v| 000000e0 97 5b d1 cc 91 92 c5 d9 05 0a |.[........| 000000ea
The etcd key should be prefixed with
k8s:enc:aescbc:v1:key1, which indicates theaescbcprovider was used to encrypt the data with thekey1encryption key. -
Deployments as they are described on Kubernetes the hard way will work
$ kubectl create deployment nginx --image=nginx deployment.apps/nginx created
List the pod created
$ kubectl get pods -l app=nginx NAME READY STATUS RESTARTS AGE nginx-8f458dc5b-86vs4 1/1 Running 0 36s
-
NodePort service will work in the following way (Depends on
nginxdeployment described in step 4)$ kubectl expose deployment nginx --port 80 --type NodePort $ NODE_PORT=$(kubectl get svc nginx \ --output=jsonpath='{range .spec.ports[0]}{.nodePort}') $ WORKER_IP=$(multipass info 'worker-1-k8s' | grep 'IPv4' | \ awk '{ print $2 }') $ curl -I "http://${WORKER_IP}:${NODE_PORT}" HTTP/1.1 200 OK Server: nginx/1.25.3 Date: Thu, 07 Dec 2023 05:28:35 GMT Content-Type: text/html Content-Length: 615 Last-Modified: Tue, 24 Oct 2023 13:46:47 GMT Connection: keep-alive ETag: "6537cac7-267" Accept-Ranges: bytes
-
You've done a kubernetes!
-
Forward hubble-ui port
POD_NAME=$(kubectl get pods -n kube-system -l k8s-app=hubble-ui \ -o jsonpath="{.items[0].metadata.name}") kubectl port-forward -n kube-system ${POD_NAME} 8080:8081
-
In your browser go to http://localhost:8080 -> pick a namespace with pods. Example below:
01-config-files/distribute-config-files.sh generates multipass-hosts and
later the bootstrap scripts append it to /etc/hosts on the controllers and
workers