Make sure we request the correct amount of ephemeral storage

emptyDir volumes use node ephemeral storage. Since we have such large emptyDir volumes for these pods, make sure we explicitly request a minimum amount of ephemeral storage so that pods are scheduled on nodes that can handle our storage needs. This is to avoid pod evictions (due to node being low on storage) and random missed reductions.
LCOGT · Dec 2, 2024 · 31c76b8 · 31c76b8
1 parent c8f13f6
commit 31c76b8
Show file tree

Hide file tree

Showing 2 changed files with 2 additions and 2 deletions.
diff --git a/helm-chart/banzai/templates/workers-large.yaml b/helm-chart/banzai/templates/workers-large.yaml
@@ -52,7 +52,7 @@ spec:
             requests:
               cpu: "0.5"
               memory: "10Gi"
-              ephemeral-storage: "128Mi"
+              ephemeral-storage: "40Gi"
             limits:
               cpu: "2"
               memory: "10Gi"

diff --git a/helm-chart/banzai/templates/workers.yaml b/helm-chart/banzai/templates/workers.yaml
@@ -52,7 +52,7 @@ spec:
             requests:
               cpu: "0.5"
               memory: "4Gi"
-              ephemeral-storage: "128Mi"
+              ephemeral-storage: "20Gi"
             limits:
               cpu: "2"
               memory: "8Gi"