feat: empty dind requests (#367)

Author: mikhail-klimko

charts/cf-runtime/Chart.yaml

@@ -1,7 +1,7 @@
 apiVersion: v2
 description: A Helm chart for Codefresh Runner
 name: cf-runtime
-version: 3.0.0
+version: 3.0.1
 keywords:
   - codefresh
   - runner
@@ -15,7 +15,7 @@ maintainers:
 annotations:
   artifacthub.io/changes: |
     - kind: added
-      description: Added runtime-environment template
+      description: Added option to specify null requests for dind
 dependencies:
   - name: cf-common
     repository: https://chartmuseum.codefresh.io/cf-common

charts/cf-runtime/README.md

@@ -1,6 +1,6 @@
 ## Codefresh Runner
 
-![Version: 3.0.0](https://img.shields.io/badge/Version-3.0.0-informational?style=flat-square)
+![Version: 3.0.1](https://img.shields.io/badge/Version-3.0.1-informational?style=flat-square)
 
 Helm chart for deploying [Codefresh Runner](https://codefresh.io/docs/docs/installation/codefresh-runner/) to Kubernetes.
 
@@ -18,6 +18,7 @@ Helm chart for deploying [Codefresh Runner](https://codefresh.io/docs/docs/insta
   - [Custom volume mounts](#custom-volume-mounts)
   - [Custom global environment variables](#custom-global-environment-variables)
   - [Volume reuse policy](#volume-reuse-policy)
+  - [Volume cleaners](#volume-cleaners)
 
 ## Prerequisites
 
@@ -61,7 +62,7 @@ helm repo update
     codefresh runner execute-test-pipeline --runtime-name <runtime-name>
     ```
 
-## Upgrade chart
+## Upgrade Chart
 
 ### To 2.x
 
@@ -275,6 +276,104 @@ Downside: More PVs to maintain and therefore higher costs.
 
 - `reuseVolumeSelector: 'codefresh-app,io.codefresh.accountName,pipeline_id,trigger'` - PV can be used only by single pipeline AND single trigger.
 
+### Volume cleaners
+
+Codefresh pipelines require disk space for:
+  * [Pipeline Shared Volume](https://codefresh.io/docs/docs/pipelines/introduction-to-codefresh-pipelines/#sharing-the-workspace-between-build-steps) (`/codefresh/volume`, implemented as [docker volume](https://docs.docker.com/storage/volumes/))
+  * Docker containers, both running and stopped
+  * Docker images and cached layers
+
+Codefresh offers two options to manage disk space and prevent out-of-space errors:
+* Use runtime cleaners on Docker images and volumes
+* [Set the minimum disk space per pipeline build volume](https://codefresh.io/docs/docs/pipelines/pipelines/#set-minimum-disk-space-for-a-pipeline-build)
+
+To improve performance by using Docker cache, Codefresh `volume-provisioner` can provision previously used disks with Docker images and pipeline volumes from previously run builds.
+
+### Types of runtime volume cleaners
+
+Docker images and volumes must be cleaned on a regular basis.
+
+* [IN-DIND cleaner](https://github.com/codefresh-io/dind/tree/master/cleaner): Deletes extra Docker containers, volumes, and images in **DIND pod**.
+* [External volume cleaner](https://github.com/codefresh-io/dind-volume-cleanup): Deletes unused **external** PVs (EBS, GCE/Azure disks).
+* [Local volume cleaner](https://github.com/codefresh-io/dind-volume-utils/blob/master/local-volumes/lv-cleaner.sh): Deletes **local** volumes if node disk space is close to the threshold.
+
+### IN-DIND cleaner
+
+**Purpose:** Removes unneeded *docker containers, images, volumes* inside Kubernetes volume mounted on the DIND pod
+
+**How it runs:** Inside each DIND pod as script
+
+**Triggered by:** SIGTERM and also during the run when disk usage > 90% (configurable)
+
+**Configured by:**  Environment Variables which can be set in Runtime Environment spec
+
+**Configuration/Logic:** [README.md](https://github.com/codefresh-io/dind/tree/master/cleaner#readme)
+
+Override `.Values.runtime.dind.env` if necessary (the following are **defaults**):
+
+```yaml
+runtime:
+  dind:
+    env:
+      CLEAN_PERIOD_SECONDS: '21600' # launch clean if last clean was more than CLEAN_PERIOD_SECONDS seconds ago
+      CLEAN_PERIOD_BUILDS: '5' # launch clean if last clean was more CLEAN_PERIOD_BUILDS builds since last build
+      IMAGE_RETAIN_PERIOD: '14400' # do not delete docker images if they have events since current_timestamp - IMAGE_RETAIN_PERIOD
+      VOLUMES_RETAIN_PERIOD: '14400' # do not delete docker volumes if they have events since current_timestamp - VOLUMES_RETAIN_PERIOD
+      DISK_USAGE_THRESHOLD: '0.8' # launch clean based on current disk usage DISK_USAGE_THRESHOLD
+      INODES_USAGE_THRESHOLD: '0.8' # launch clean based on current inodes usage INODES_USAGE_THRESHOLD
+```
+
+### External volumes cleaner
+
+**Purpose:** Removes unused *kubernetes volumes and related backend volumes*
+
+**How it runs:** Runs as `dind-volume-cleanup` CronJob. Installed in case the Runner uses non-local volumes `.Values.storage.backend != local`
+
+**Triggered by:** CronJob every 10min (configurable)
+
+**Configuration:**
+
+Set `codefresh.io/volume-retention` for dinds' PVCs:
+
+```yaml
+runtime:
+  dind:
+    pvcs:
+      - name: dind
+        ...
+        annotations:
+          codefresh.io/volume-retention: 7d
+```
+
+Or override environment variables for `dind-volume-cleanup` cronjob:
+
+```yaml
+volumeProvisioner:
+  dind-volume-cleanup:
+    env:
+      RETENTION_DAYS: 7   # clean volumes that were last used more than `RETENTION_DAYS` (default is 4) ago
+```
+
+### Local volumes cleaner
+
+**Purpose:** Deletes local volumes when node disk space is close to the threshold
+
+**How it runs:** Runs as `dind-lv-monitor` DaemonSet. Installed in case the Runner uses local volumes `.Values.storage.backend == local`
+
+**Triggered by:** Disk space usage or inode usage that exceeds thresholds (configurable)
+
+**Configuration:**
+
+Override environment variables for `dind-lv-monitor` daemonset:
+
+```yaml
+volumeProvisioner:
+  dind-lv-monitor:
+    env:
+      KB_USAGE_THRESHOLD: 60  # default 80 (percentage)
+      INODE_USAGE_THRESHOLD: 60  # default 80
+```
+
 ## Requirements
 
 | Repository | Name | Version |
@@ -371,7 +470,7 @@ Downside: More PVs to maintain and therefore higher costs.
 | runner.tolerations | list | `[]` | Set tolerations |
 | runner.updateStrategy | object | `{"type":"RollingUpdate"}` | Upgrade strategy |
 | runtime | object | See below | Set runtime parameters |
-| runtime.dind | object | `{"affinity":{},"env":{},"image":{"registry":"quay.io","repository":"codefresh/dind","tag":"20.10.18-1.25.7"},"nodeSelector":{},"podAnnotations":{},"pvcs":[{"name":"dind","reuseVolumeSelector":"codefresh-app,io.codefresh.accountName","reuseVolumeSortOrder":"pipeline_id","storageClassName":"{{ include \"dind-volume-provisioner.storageClassName\" . }}","volumeSize":"16Gi"}],"resources":{"limits":{"cpu":"400m","memory":"800Mi"},"requests":{"cpu":"400m","memory":"800Mi"}},"schedulerName":"","serviceAccount":"codefresh-engine","tolerations":[],"userAccess":true,"userVolumeMounts":{},"userVolumes":{}}` | Parameters for DinD (docker-in-docker) pod (aka "runtime" pod). |
+| runtime.dind | object | `{"affinity":{},"env":{},"image":{"registry":"quay.io","repository":"codefresh/dind","tag":"20.10.18-1.25.7"},"nodeSelector":{},"podAnnotations":{},"pvcs":[{"name":"dind","reuseVolumeSelector":"codefresh-app,io.codefresh.accountName","reuseVolumeSortOrder":"pipeline_id","storageClassName":"{{ include \"dind-volume-provisioner.storageClassName\" . }}","volumeSize":"16Gi"}],"resources":{"limits":{"cpu":"400m","memory":"800Mi"},"requests":null},"schedulerName":"","serviceAccount":"codefresh-engine","tolerations":[],"userAccess":true,"userVolumeMounts":{},"userVolumes":{}}` | Parameters for DinD (docker-in-docker) pod (aka "runtime" pod). |
 | runtime.dind.affinity | object | `{}` | Set affinity |
 | runtime.dind.env | object | `{}` | Set additional env vars. |
 | runtime.dind.image | object | `{"registry":"quay.io","repository":"codefresh/dind","tag":"20.10.18-1.25.7"}` | Set dind image. |
@@ -382,7 +481,7 @@ Downside: More PVs to maintain and therefore higher costs.
 | runtime.dind.pvcs[0].reuseVolumeSelector | string | `"codefresh-app,io.codefresh.accountName"` | PV reuse selector. Ref: https://codefresh.io/docs/docs/installation/codefresh-runner/#volume-reuse-policy |
 | runtime.dind.pvcs[0].storageClassName | string | `"{{ include \"dind-volume-provisioner.storageClassName\" . }}"` | PVC storage class name. Change ONLY if you need to use storage class NOT from Codefresh volume-provisioner |
 | runtime.dind.pvcs[0].volumeSize | string | `"16Gi"` | PVC size. |
-| runtime.dind.resources | object | `{"limits":{"cpu":"400m","memory":"800Mi"},"requests":{"cpu":"400m","memory":"800Mi"}}` | Set dind resources. |
+| runtime.dind.resources | object | `{"limits":{"cpu":"400m","memory":"800Mi"},"requests":null}` | Set dind resources. |
 | runtime.dind.schedulerName | string | `""` | Set scheduler name. |
 | runtime.dind.serviceAccount | string | `"codefresh-engine"` | Set service account for pod. |
 | runtime.dind.tolerations | list | `[]` | Set tolerations. |

charts/cf-runtime/README.md.gotmpl

@@ -18,6 +18,7 @@ Helm chart for deploying [Codefresh Runner](https://codefresh.io/docs/docs/insta
   - [Custom volume mounts](#custom-volume-mounts)
   - [Custom global environment variables](#custom-global-environment-variables)
   - [Volume reuse policy](#volume-reuse-policy)
+  - [Volume cleaners](#volume-cleaners)
 
 ## Prerequisites
 
@@ -61,7 +62,7 @@ helm repo update
     codefresh runner execute-test-pipeline --runtime-name <runtime-name>
     ```
 
-## Upgrade chart
+## Upgrade Chart
 
 ### To 2.x
 
@@ -275,6 +276,103 @@ Downside: More PVs to maintain and therefore higher costs.
 
 - `reuseVolumeSelector: 'codefresh-app,io.codefresh.accountName,pipeline_id,trigger'` - PV can be used only by single pipeline AND single trigger.
 
+### Volume cleaners
+
+Codefresh pipelines require disk space for:
+  * [Pipeline Shared Volume](https://codefresh.io/docs/docs/pipelines/introduction-to-codefresh-pipelines/#sharing-the-workspace-between-build-steps) (`/codefresh/volume`, implemented as [docker volume](https://docs.docker.com/storage/volumes/))
+  * Docker containers, both running and stopped
+  * Docker images and cached layers
+
+Codefresh offers two options to manage disk space and prevent out-of-space errors:
+* Use runtime cleaners on Docker images and volumes
+* [Set the minimum disk space per pipeline build volume](https://codefresh.io/docs/docs/pipelines/pipelines/#set-minimum-disk-space-for-a-pipeline-build)
+
+To improve performance by using Docker cache, Codefresh `volume-provisioner` can provision previously used disks with Docker images and pipeline volumes from previously run builds.
+
+### Types of runtime volume cleaners
+
+Docker images and volumes must be cleaned on a regular basis.
+
+* [IN-DIND cleaner](https://github.com/codefresh-io/dind/tree/master/cleaner): Deletes extra Docker containers, volumes, and images in **DIND pod**.
+* [External volume cleaner](https://github.com/codefresh-io/dind-volume-cleanup): Deletes unused **external** PVs (EBS, GCE/Azure disks).
+* [Local volume cleaner](https://github.com/codefresh-io/dind-volume-utils/blob/master/local-volumes/lv-cleaner.sh): Deletes **local** volumes if node disk space is close to the threshold.
+
+### IN-DIND cleaner
+
+**Purpose:** Removes unneeded *docker containers, images, volumes* inside Kubernetes volume mounted on the DIND pod
+
+**How it runs:** Inside each DIND pod as script
+
+**Triggered by:** SIGTERM and also during the run when disk usage > 90% (configurable)
+
+**Configured by:**  Environment Variables which can be set in Runtime Environment spec
+
+**Configuration/Logic:** [README.md](https://github.com/codefresh-io/dind/tree/master/cleaner#readme)
+
+Override `.Values.runtime.dind.env` if necessary (the following are **defaults**):
+
+```yaml
+runtime:
+  dind:
+    env:
+      CLEAN_PERIOD_SECONDS: '21600' # launch clean if last clean was more than CLEAN_PERIOD_SECONDS seconds ago
+      CLEAN_PERIOD_BUILDS: '5' # launch clean if last clean was more CLEAN_PERIOD_BUILDS builds since last build
+      IMAGE_RETAIN_PERIOD: '14400' # do not delete docker images if they have events since current_timestamp - IMAGE_RETAIN_PERIOD
+      VOLUMES_RETAIN_PERIOD: '14400' # do not delete docker volumes if they have events since current_timestamp - VOLUMES_RETAIN_PERIOD
+      DISK_USAGE_THRESHOLD: '0.8' # launch clean based on current disk usage DISK_USAGE_THRESHOLD
+      INODES_USAGE_THRESHOLD: '0.8' # launch clean based on current inodes usage INODES_USAGE_THRESHOLD
+```
+
+### External volumes cleaner
+
+**Purpose:** Removes unused *kubernetes volumes and related backend volumes*
+
+**How it runs:** Runs as `dind-volume-cleanup` CronJob. Installed in case the Runner uses non-local volumes `.Values.storage.backend != local`
+
+**Triggered by:** CronJob every 10min (configurable)
+
+**Configuration:**
+
+Set `codefresh.io/volume-retention` for dinds' PVCs:
+
+```yaml
+runtime:
+  dind:
+    pvcs:
+      - name: dind
+        ...
+        annotations:
+          codefresh.io/volume-retention: 7d
+```
+
+Or override environment variables for `dind-volume-cleanup` cronjob:
+
+```yaml
+volumeProvisioner:
+  dind-volume-cleanup:
+    env:
+      RETENTION_DAYS: 7   # clean volumes that were last used more than `RETENTION_DAYS` (default is 4) ago
+```
+
+### Local volumes cleaner
+
+**Purpose:** Deletes local volumes when node disk space is close to the threshold
+
+**How it runs:** Runs as `dind-lv-monitor` DaemonSet. Installed in case the Runner uses local volumes `.Values.storage.backend == local`
+
+**Triggered by:** Disk space usage or inode usage that exceeds thresholds (configurable)
+
+**Configuration:**
+
+Override environment variables for `dind-lv-monitor` daemonset:
+
+```yaml
+volumeProvisioner:
+  dind-lv-monitor:
+    env:
+      KB_USAGE_THRESHOLD: 60  # default 80 (percentage)
+      INODE_USAGE_THRESHOLD: 60  # default 80
+```
 
 {{ template "chart.requirementsSection" . }}
 

charts/cf-runtime/templates/runtime/runtime-env-spec-tmpl.yaml

@@ -94,8 +94,13 @@ dockerDaemonScheduler:
   {{- end }}
   defaultDindResources:
   {{- with $dindContext.resources }}
+  {{- if not .requests }}
+    limits: {{- toYaml .limits | nindent 6 }}
+    requests: null
+  {{- else }}
   {{- toYaml . | nindent 4 }}
   {{- end }}
+  {{- end }}
   {{- with $dindContext.terminationGracePeriodSeconds }}
   terminationGracePeriodSeconds: {{ . }}
   {{- end }}

charts/cf-runtime/tests/runtime/runtime_test.yaml

@@ -128,9 +128,7 @@ tests:
                 limits:
                   cpu: 1000m
                   memory: 2048Mi
-                requests:
-                  cpu: 1000m
-                  memory: 2048Mi
+                requests: null
               userVolumeMounts:
                 my-cert:
                   mountPath: /etc/ssl/cert

charts/cf-runtime/tests/runtime/runtime_values.yaml

@@ -3,9 +3,7 @@ runtime:
     image:
       tag: tagoverride
     resources:
-      requests:
-        cpu: 1000m
-        memory: 2048Mi
+      requests: null
       limits:
         cpu: 1000m
         memory: 2048Mi

charts/cf-runtime/values.yaml

@@ -379,9 +379,7 @@ runtime:
       tag: 20.10.18-1.25.7
     # -- Set dind resources.
     resources:
-      requests:
-        cpu: 400m
-        memory: 800Mi
+      requests: null
       limits:
         cpu: 400m
         memory: 800Mi