Cubienetes: A Kubernetes Cluster on Cubieboard2s

This is a project to build a home 5-node Kubernetes cluster using Cubieboards.

Parts

• 5 x Cubieboard2 £48.95
• 1 x NETGEAR GS108UK 8-Port Gigabit Ethernet Unmanaged Switch £24.97
• 1 x Anker 40W 5-Port USB Charger
• 5 x Short Black 0.25M Ethernet Cable £1.81
• 1 x Kingston MobileLite G4 USB 3.0 Multi Card Reader £7.12
• 2 x 1000mm M3 threaded rod £2.54
• 2 x M3 Full Nut (50 Pack) £1.40
• 2 x M3 Washer 3.2mm (50 Pack) £1.65
• 10 x Custom 3D-printed HD brackets £3.50
• 5 x Transcend 128GB SATA III SSD360 128GB £56.50

Assembly

First versions of the hard disk brackets didn't quite fit:

I did a full build to get a handle on the cabling too:

Problems were:

• The SSD mounting hole pitch was off by 0.5mm.
• The walls around the pillars meant the distance between the pillars was too small.
• The bore of all the holes (3mm) was too tight for M3 threaded rod and needed drilling out. Surprisingly the Shapeways white plastic took this quite well.

So I created & new design and ordered that:

And with the router & PSU double-sided-sticky taped on, and cables all cable-tied:

Software

Operating System

• Used Debian Jessie from https://www.armbian.com/cubieboard-2/
• Use Keka and Etcher to decompress / write the image.
• Use the latest kernel you can - Docker needs it.

Manual setup

For ansible to work, I setup password-less ssh:

mkdir .ssh
chmod 700 ~/.ssh
echo "<REDACTED>" > ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys

I did this manually. I also statically assigned IP addresses 192.168.0.1-5 to the Cubieboards manually. The rest was configured with ansible.

Automated setup

I designated the first Cubieboard as the "master", and set it up to do NAT routing & DNS as it also has a WiFi adapter.

ansible-playbook --inventory-file=inventory --user=root custom-playbooks/networking.yaml

The disks got LVM & ext4 installed and mounted at /mnt/countainers:

ansible-playbook --inventory-file=inventory --user=root custom-playbooks/disks.yaml

And the hostnames were all set and put in /etc/hosts:

ansible-playbook --inventory-file=inventory --user=root custom-playbooks/hostname.yaml

Docker Daemon

• Used Hypriot's instructions
• https://packagecloud.io/app/Hypriot/Schatzkiste/search?filter=all&q=docker-hypriot&dist=debian%2Fjessie

ansible-playbook --inventory-file=inventory --user=root custom-playbooks/docker.yaml