Skip to content

gen2-v20230906.0
Compare
Choose a tag to compare
@AkihiroSuda AkihiroSuda released this 05 Sep 20:55
· 42 commits to master since this release
gen2-v20230906.0
This tag was signed with the committer’s verified signature.
AkihiroSuda Akihiro Suda
GPG key ID: 49524C6F9F638F1A
Learn about vigilant mode.
9c454e1
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

This is the first release of "Usernetes Generation 2" (#287)

Usernetes: Kubernetes without the root privileges (Generation 2)

Usernetes (Gen2) deploys a Kubernetes cluster inside Rootless Docker,
so as to mitigate potential container-breakout vulnerabilities.

Note

Usernetes (Gen2) has significantly diverged from the original Usernetes (Gen1),
which did not require Rootless Docker to be installed on hosts.

See the gen1 branch for
the original Usernetes (Gen1).

Usernetes (Gen2) is similar to Rootless kind and Rootless minikube,
but Usernetes (Gen 2) supports creating a cluster with multiple hosts.

Components

  • Cluster configuration: kubeadm
  • CRI: containerd
  • OCI: runc
  • CNI: Flannel

Requirements

Note

Using Ubuntu 22.04 hosts is recommended.

curl -o install.sh -fsSL https://get.docker.com
sudo sh install.sh
dockerd-rootless-setuptool.sh install
  • systemd lingering:
sudo loginctl enable-linger $(whoami)
  • cgroup v2 delegation:
sudo mkdir -p /etc/systemd/system/[email protected]

cat <<EOF | sudo tee /etc/systemd/system/[email protected]/delegate.conf
[Service]
Delegate=cpu cpuset io memory pids
EOF

sudo systemctl daemon-reload
  • Kernel modules:
sudo modprobe vxlan

Usage

See make help.

# Bootstrap a cluster
make up
make kubeadm-init
make install-flannel

# Enable kubectl
make kubeconfig
export KUBECONFIG=$(pwd)/kubeconfig
kubectl get pods -A

# Multi-host
make join-command
scp join-command another-host:~/usernetes
ssh another-host make -C ~/usernetes up kubeadm-join

# Debug
make logs
make shell
make down-v
kubectl taint nodes --all node-role.kubernetes.io/control-plane-

Limitations

  • Node ports cannot be exposed automatically. Edit docker-compose.yaml for exposing additional node ports.
  • Most of host files are not visible with hostPath mounts. Edit docker-compose.yaml for mounting additional files.
  • Some volume drivers such as nfs do not work.

Advanced topics

  • Although Usernetes (Gen2) is designed to be used with Rootless Docker, it should work with the regular "rootful" Docker too.
    This might be useful for some people who are looking for "multi-host" version of kind and minikube.     (Support for "rootful" is currently broken: #297)
Assets 2
Loading