This project aims to create a Kubernetes controller that watches Pod Disruption Budgets (PDBs) and Deployments, alongside a webhook that monitors pod eviction requests initiated by a client, logs them, and communicates them back to the controller. The controller will dynamically adjust resources in response to the constraints by the PDB during disruptions.
- Controller: Continuously watch PDBs and their associated pods to detect when evictions are blocked. Ensure the controller accurately identifies the disruption state without causing unnecessary scaling actions.
- Webhook: Create an admission webhook to intercept pod eviction requests to determine if evictions are being blocked due to DisruptionsAllowed being zero. As DisruptionsAllowed could be zero as a desired state, annotate accordingly.
- Controller: Automatically scale Deployments or StatefulSets when necessary, specifically when evictions are blocked due to DisruptionsAllowed being zero and an actual eviction attempt is detected from the webhook.
- Webhook: Provide real-time information about eviction attempts, allowing the controller to make immediate scaling decisions, and to prevent scaling decisions that are against what the author of deployment and PDB is trying to tell you.
- Controller: Produce events or update the status of PDBs to notify cluster administrators about scaling actions taken. Generate events when scaling occurs and update annotations to reflect current states.
- Webhook: Log events or update the status of PDBs or pods when eviction attempts are blocked.
- Controller: Avoid unnecessary scaling actions that could lead to resource wastage or application downtime.
- Webhook: Provide detailed insights into eviction attempts and PDB statuses, ensuring that the controller only scales resources when truly necessary.
Follow the steps below to install and set up the k8s-pdb-autoscaler in your Kubernetes cluster:
- A Kubernetes cluster
kubectlconfigured to interact with your cluster- Docker installed on your local machine
First, clone the repository and navigate to the project directory:
git clone https://github.com/your-repo/k8s-pdb-autoscaler.git
cd k8s-pdb-autoscalerMake sure the script is executable:
chmod +x install.shExecute the installation script:
./install.shRun
kubectl get podsTo verify controller and webhook have been deployed without any issues. You should see the controller and webhook pods running. If they are not running, check the logs for any errors:
kubectl logs <controller-pod-name>
kubectl logs <webhook-pod-name>Now run the autodeploy.sh script so the controller and webhook can communicated with the deployments within the cluster within the default namespace, this script will create PodDisruptionBudgets (PDBs) and PDBWatchers for all deployments in the default namespace. It is customizable to fit your needs.
Make sure the script is executable
bash chmod +x autodeploy.sh
Run the Script Execute the script to create and apply the PDBs and PDBWatchers:
./autodeploy.shAfter running the scripts, you need to ensure that both the controller and webhook are working as expected.
Simulate a Pod Eviction You can manually attempt to evict a pod and check if the webhook logs the eviction request and if the controller reacts by adjusting the deployment's scale based on the PDB:
Attempt to Evict a Pod:
kubectl delete pod <pod-name> -n <namespace>Check Webhook Logs:
Verify that the webhook intercepted the eviction request:
kubectl logs <webhook-pod-name> -n <namespace>Check Controller Logs: Verify that the controller took appropriate action, such as scaling a deployment:
kubectl logs <controller-pod-name> -n <namespace>Check Deployment and PDB Status:
Ensure that the PDB status and deployment replicas have been updated accordingly:
kubectl get pdb -n <namespace>
kubectl get deployment -n <namespace>