A configurable, flexible, and consistent local Kubernetes cluster (or clusters) with support for most things that you would expect in a cluster such as LoadBalancer and persistent storage out of the box.
This uses Vagrant and kubeadm to spin up a local Kuberentes cluster. It then installs metallb for a local LoadBalancer and Longhorn for persistent volumes. It exposes several of the more common things one might like to configure through the settings.yaml file such as Kubernetes version and the configuration of the virtual machines.
This project is meant to follow the Keep It Simple Stupid philosophy. I want the project to be organized in a way and function in a way that it's dead easy for people to use and modify it in a way that works best for them.
This means I will do my best to ensure well commented code, no overly convoluted code, and no strange dependencies or libraries.
Why did I make this:
- I needed a local Kubernetes cluster running on Virtual Machines instead of within containers because I needed
open-iscsiand mount propagation which doesn't work properly within containers - I wanted something simple that I understood instead of having to learn another tool exclusively for local development, and then figure out how to translate that to my homelab cluster. Using Vagrant it's simply Virtual Machines.
What about Minikube, KiND, etc.
They're great, and may be the right fit for your needs. I have used them and they work great, they just weren't what I was looking for. Also, as mentioned earlier, I didn't want to learn another tool.
The whole point of the project is to spin up a Kubernetes cluster.
Used to provision the virtual machines using VirtualBox
Makes heavy use of bash to provision the Virtual Machines after they are created.
The cluster is built using kubeadm, using containerd as a CRI, and Calico for pod networking. It then adds the following additional software if configured to do so.
metallb
Metallb is optionally installed to provide a local load balancer into the cluster.
Longhorn
Longhorn is optionally installed to provide persistent volumes with support for ReadWriteMany.
This project requires only that you have Vagrant and VirtualBox installed on your machine, and have enough resources available to run your configured number of machines with the amount of allocated resources.
With the default configuration it builds 1 master node and 1 worker node each with 2Gb of RAM and 2 cores which is the suggested minimum from the Kubernetes project.
Install VirtualBox from their website, Chocolatey on Windows, from your package manager on Linux, or with Brew on Mac.
Windows
choco install virtualbox
Linux (Debian/Ubuntu)
apt install virtualbox
Mac
brew install virtualbox
Install Vagrant using Chocolatey on Windows, from your package manager on Linux, or with Brew on Mac.
Windows
choco install vagrantLinux (Debian/Ubuntu)
apt install vagrantMac
brew install vagrant-
Clone the repo
git clone https://github.com/danhenrydev/k8s-vagrant.git
-
Modify the configuration file at
cluster-settings.yamlto your liking. It comes with sane defaults. -
Change into the directory and run
vagrant upcd k8s-vagrant vagrant up -
Copy the kubectl config from
generated/configinto your local.kubeor set the environment variable to access the cluster from your local machine.mkdir ~/.kube cp generated/config ~/.kube/config kubectl get nodes ### OR ### KUBECONFIG=generated/config kubectl get nodes
When you aren't using your cluster it makes sense to shut it down, but not destroy it as it can take a significant amount of time to rebuild in the future. Fortunately Vagrant gives us this option.
Pause your cluster
cd k8s-vagrant
vagrant suspend
Resume your cluster
cd k8s-vagrant
vagrant resume
When you're finished with your cluster, or you've broken it. You can destroy your cluster to re-initialize it for a fresh start.
cd k8s-vagrant
vagrant destroy
Although the project seems to be working well for me as is, there are some things that I would like to add in the future that I haven't gotten around to yet. Here's what I've thought about, in no specific order.
- Add a Traefik Ingress option
- Add support for VmWare Workstation
- Add support for QEMU
-
Pretty up the bash scripts and output - Generate a file at the end with relevant information
- Declutter the Vagrantfile
- Settings sanity check before running vagrantfile
If you have a suggestion, please reach out to me via Twitter and I'll see what I can do to get it added.
If you run into a problem or bug and it's not something you can fix, please feel free to create an issue on Github and I will see what I can do to get it fixed.
If you run into a problem or have an idea, and it IS something you can fix, please create a PR. When doing so, please keep in mind the KISS principle as mentioned in the About this Project section.
Distributed under the GPLv2 License. See LICENSE.md for more information.
Daniel Henry - @danhenrydev
Project Link: https://github.com/danhenrydev/k8s-vagrant
Obviously I'm not the first to make something like this, so thank you to everyone that has solved some of these problems in the past. Some specific projects that helped me a lot below.