This is a fork from CARV-ICS-FORTH/HPK. The repo has been adapted to run with podman-hpc and Slurm, which are in use at NERSC.
The commands to install and run remain the same except for make run-kubemaster which has been replaced by make run-podman.
The Kubernetes control plane is now a new container adapted from the original.
This modified version of High-Performance Kubernetes (HPK), allows HPC users to run their own private "mini Clouds" on a typical HPC cluster, which uses a single container to run the Kubernetes control plane and a Virtual Kubelet Provider implementation to translate container lifecycle management commands from Kubernetes-native to Slurm/Podman-hpc.
In contrast to a typical Kubernetes installation at the Cloud:
- HPK uses a pass-through scheduler, which assigns all pods to the single
hpk-kubeletthat represents the cluster. In practice, this means that all scheduling is delegated to Slurm. - All Kubernetes services are converted to headless. This avoids the need for internal, virtual cluster IPs that would need special handling at the network level. As a side effect, HPK services that map to multiple pods are load-balanced at the DNS level if clients support it.
HPK is a continuation of the KNoC project, a Virtual Kubelet Provider implementation that can be used to bridge Kubernetes and HPC environments.
Compile the hpk-kubelet using make.
make buildThen you need to start the Kubernetes Master and hpk-kubelet seperately.
To run the Kubernetes Master:
make run-podmanOnce the master is up and running, you can start the hpk-kubelet:
make run-kubeletNow you can configure and use kubectl:
export KUBE_PATH=~/.k8sfs/kubernetes/
export KUBECONFIG=${KUBE_PATH}/admin.conf
kubectl get nodesIn case that you experience DNS issues, you should retry starting the Kubernetes Master with:
export EXTERNAL_DNS=<your dns server>
make run-kubemaster
The above command will set CoreDNS to forward requests for external names to your DNS server.