Experimental Vector chart (#24)

* Initial commit for experimental Vector chart Signed-off-by: Spencer Gilbert <[email protected]> * Fix chart lint Signed-off-by: Spencer Gilbert <[email protected]> * Set version to 0.1.0-alpha.0 Signed-off-by: Spencer Gilbert <[email protected]> * Add helm-docs check to CI update vector-experimental readme Signed-off-by: Spencer Gilbert <[email protected]> * Update links to datadog/helm-charts main branch Signed-off-by: Spencer Gilbert <[email protected]> * Add configurable updateStrategy Signed-off-by: Spencer Gilbert <[email protected]> * Add initial stateless-aggregator deployment Signed-off-by: Spencer Gilbert <[email protected]> * Add stateless role for existing aggregator conditionals Signed-off-by: Spencer Gilbert <[email protected]> * Note old agent and aggregator charts as deprecated Signed-off-by: Spencer Gilbert <[email protected]> * Updates for aggregator roles Signed-off-by: Spencer Gilbert <[email protected]> * Initial addition of HAProxy load balancer Signed-off-by: Spencer Gilbert <[email protected]> * Move experimental chart to be just vector Signed-off-by: Spencer Gilbert <[email protected]> * Fix persistence for agents and aggregators Signed-off-by: Spencer Gilbert <[email protected]> * Polish artifacthub config and bump haproxy tag Signed-off-by: Spencer Gilbert <[email protected]> * Fix serviceAccount comments Signed-off-by: Spencer Gilbert <[email protected]> * Remove api ports from service Signed-off-by: Spencer Gilbert <[email protected]> * Add containerPorts helper Signed-off-by: Spencer Gilbert <[email protected]> * Add ci for persistent agent config Signed-off-by: Spencer Gilbert <[email protected]> * Revert logic around 0 port services, too brittle Signed-off-by: Spencer Gilbert <[email protected]> * Add prometheus-exporter port to default configs Signed-off-by: Spencer Gilbert <[email protected]> * Rename prom-exporter, and trim port names for containers Signed-off-by: Spencer Gilbert <[email protected]> * Rename prometheus sinks/sources for clarity and brevity Signed-off-by: Spencer Gilbert <[email protected]> * Note default agent config as currently broken Signed-off-by: Spencer Gilbert <[email protected]> * Mark chart as pre-release on artifacthub Signed-off-by: Spencer Gilbert <[email protected]>
vectordotdev · Sep 8, 2021 · 687e6e8 · 687e6e8
1 parent 307a754
commit 687e6e8
Show file tree

Hide file tree

Showing 35 changed files with 1,623 additions and 5 deletions.
diff --git a/.github/helm-docs.sh b/.github/helm-docs.sh
@@ -0,0 +1,14 @@
+#!/bin/bash
+# Credit: https://github.com/DataDog/helm-charts/blob/main/.github/helm-docs.sh
+set -euo pipefail
+
+HELM_DOCS_VERSION="1.5.0"
+OS=$(uname)
+
+# install helm-docs
+curl --silent --show-error --fail --location --output /tmp/helm-docs.tar.gz https://github.com/norwoodj/helm-docs/releases/download/v"${HELM_DOCS_VERSION}"/helm-docs_"${HELM_DOCS_VERSION}"_${OS}_x86_64.tar.gz
+tar -xf /tmp/helm-docs.tar.gz helm-docs
+
+# validate docs
+./helm-docs
+git diff --exit-code
diff --git a/.github/kubeval.sh b/.github/kubeval.sh
@@ -1,5 +1,5 @@
 #!/bin/bash
-# Credit: https://github.com/DataDog/helm-charts/blob/master/.github/kubeval.sh
+# Credit: https://github.com/DataDog/helm-charts/blob/main/.github/kubeval.sh
 set -euo pipefail
 
 KUBEVAL_VERSION="0.16.1"

diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml
@@ -1,4 +1,4 @@
-# Credit: https://github.com/DataDog/helm-charts/blob/master/.github/workflows/ci.yaml
+# Credit: https://github.com/DataDog/helm-charts/blob/main/.github/workflows/ci.yaml
 name: Lint and Test Charts
 
 on:
@@ -45,6 +45,18 @@ jobs:
       - name: Run chart-testing (lint)
         run: ct lint --config .github/ct.yaml
 
+  lint-docs:
+    runs-on: ubuntu-latest
+    needs:
+      - changed
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v2
+        with:
+          fetch-depth: 0
+      - name: Run helm-docs
+        run: .github/helm-docs.sh
+
   kubeval-chart:
     runs-on: ubuntu-latest
     needs:

diff --git a/.github/workflows/release.yaml b/.github/workflows/release.yaml
@@ -1,4 +1,4 @@
-# Credit: https://github.com/DataDog/helm-charts/blob/master/.github/workflows/release.yaml
+# Credit: https://github.com/DataDog/helm-charts/blob/main/.github/workflows/release.yaml
 name: Release Charts
 
 on:

diff --git a/.helmdocsignore b/.helmdocsignore
@@ -0,0 +1,3 @@
+charts/vector-agent
+charts/vector-aggregator
+charts/vector-shared
diff --git a/README.md b/README.md
@@ -2,8 +2,8 @@
 
 Official Helm charts for Vector. Currently supported:
 - [Vector](charts/vector/README.md) (vector/vector)
-- [Vector Agents](charts/vector-agent/README.md) (vector/vector-agent)
-- [Vector Aggregators](charts/vector-aggregator/README.md) (vector/vector-aggregator)
+- [Vector Agents](charts/vector-agent/README.md) (vector/vector-agent) - **Deprecated**
+- [Vector Aggregators](charts/vector-aggregator/README.md) (vector/vector-aggregator) - **Deprecated**
 
 # How to use the Vector Helm Repository
 

diff --git a/charts/vector/.helmignore b/charts/vector/.helmignore
@@ -0,0 +1,23 @@
+# Patterns to ignore when building packages.
+# This supports shell glob matching, relative path matching, and
+# negation (prefixed with !). Only one pattern per line.
+.DS_Store
+# Common VCS dirs
+.git/
+.gitignore
+.bzr/
+.bzrignore
+.hg/
+.hgignore
+.svn/
+# Common backup files
+*.swp
+*.bak
+*.tmp
+*.orig
+*~
+# Various IDEs
+.project
+.idea/
+*.tmproj
+.vscode/
diff --git a/charts/vector/Chart.yaml b/charts/vector/Chart.yaml
@@ -0,0 +1,32 @@
+apiVersion: v2
+name: vector
+version: "0.1.0-alpha.0"
+kubeVersion: ">=1.15.0"
+description: A lightweight, ultra-fast tool for building observability pipelines
+type: application
+keywords:
+  - vector
+  - events
+  - logs
+  - metrics
+  - observability
+  - stream-processing
+home: https://vector.dev/
+sources:
+  - https://github.com/timberio/vector/
+maintainers:
+  - name: Datadog
+    email: [email protected]
+icon: https://vector.dev/press/vector-icon.svg
+appVersion: "0.16.1-distroless-libc"
+annotations:
+  artifacthub.io/images: |
+    - name: vector
+      image: timberio/vector:0.16.1-distroless-libc
+    - name: haproxy
+      image: haproxytech/haproxy-alpine:2.4.4
+  artifacthub.io/license: MPL-2.0
+  artifacthub.io/links: |
+    - name: Chart Source
+      url: https://github.com/timberio/helm-charts
+  artifacthub.io/prerelease: "true"
diff --git a/charts/vector/README.md b/charts/vector/README.md
@@ -0,0 +1,128 @@
+# Vector
+
+![Version: 0.1.0-alpha.0](https://img.shields.io/badge/Version-0.1.0--alpha.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 0.16.1-distroless-libc](https://img.shields.io/badge/AppVersion-0.16.1--distroless--libc-informational?style=flat-square)
+
+[Vector](https://vector.dev/) is a high-performance, end-to-end observability data pipeline that puts you in control of your observability data. Collect, transform, and route all your logs, metrics, and traces to any vendors you want today and any other vendors you may want tomorrow. Vector enables dramatic cost reduction, novel data enrichment, and data security where you need it, not where is most convenient for your vendors.
+
+## How to use Vector Helm repository
+
+You need to add this repository to your Helm repositories:
+
+```
+helm repo add vector https://helm.vector.dev
+helm repo update
+```
+
+## Requirements
+
+Kubernetes: `>=1.15.0`
+
+## Quick start
+
+By default, Vector runs in the Aggregator role. It can alternatively run as a DaemonSet for the Agent role.
+
+### Installing the Vector chart
+
+To install the chart with the release name `<RELEASE_NAME>` run:
+
+```bash
+helm install --name <RELEASE_NAME> \
+  vector/vector
+```
+
+### Upgrading
+
+TODO
+
+### Uninstalling the Chart
+
+To uninstall/delete the `<RELEASE_NAME>` deployment:
+
+```bash
+helm delete <RELEASE_NAME> --purge
+```
+
+The command removes all the Kubernetes components associated with the chart and deletes the release.
+
+## Configuration
+
+1. Using our [`default-values.yaml`](values.yaml) configuration file as a reference, create your own `values.yaml`.
+1. Upgrade the Vector chart with your new `values.yaml` file:
+
+```bash
+helm upgrade -f values.yaml <RELEASE_NAME> vector/vector
+```
+
+**Vector recommends that your `values.yaml` only contain values that need to be overridden, as it allows a smoother experience when upgrading chart versions.**
+
+See the [All configuration options](#all-configuration-options) section to discover all possibilities offered by the Vector chart.
+
+## All configuration options
+
+The following table lists the configurable parameters of the Vector chart and their default values. Specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. For example,
+
+```bash
+helm install --name <RELEASE_NAME> \
+  --set role=Agent \
+  vector/vector
+```
+
+## Values
+
+| Key | Type | Default | Description |
+|-----|------|---------|-------------|
+| affinity | object | `{}` | Allow Vector to schedule using affinity rules |
+| autoscaling.customMetric | object | `{}` |  |
+| autoscaling.enabled | bool | `false` | Enabled autoscaling for the Stateless-Aggregator |
+| autoscaling.maxReplicas | int | `10` |  |
+| autoscaling.minReplicas | int | `1` |  |
+| autoscaling.targetCPUUtilizationPercentage | int | `80` |  |
+| customConfig | object | `{}` | Override Vector's default configs, if used **all** options need to be specified |
+| haproxy.affinity | object | `{}` |  |
+| haproxy.autoscaling.customMetric | object | `{}` |  |
+| haproxy.autoscaling.enabled | bool | `false` | Enabled autoscaling for the HAProxy load balancer |
+| haproxy.autoscaling.maxReplicas | int | `10` |  |
+| haproxy.autoscaling.minReplicas | int | `1` |  |
+| haproxy.autoscaling.targetCPUUtilizationPercentage | int | `80` |  |
+| haproxy.enabled | bool | `false` | If true, create a HAProxy load balancer |
+| haproxy.fullnameOverride | string | `""` |  |
+| haproxy.image.pullPolicy | string | `"IfNotPresent"` |  |
+| haproxy.image.repository | string | `"haproxytech/haproxy-alpine"` |  |
+| haproxy.image.tag | string | `"2.4.4"` |  |
+| haproxy.imagePullSecrets | list | `[]` |  |
+| haproxy.nameOverride | string | `""` |  |
+| haproxy.nodeSelector | object | `{}` |  |
+| haproxy.podAnnotations | object | `{}` |  |
+| haproxy.podSecurityContext | object | `{}` |  |
+| haproxy.replicas | int | `1` | Set the number of pods to create |
+| haproxy.resources | object | `{}` |  |
+| haproxy.securityContext | object | `{}` |  |
+| haproxy.service.type | string | `"ClusterIP"` |  |
+| haproxy.serviceAccount.create | bool | `true` | If true, create ServiceAccount |
+| haproxy.serviceAccount.name | string | `nil` | The name of the ServiceAccount to use. |
+| haproxy.terminationGracePeriodSeconds | int | `60` |  |
+| haproxy.tolerations | list | `[]` |  |
+| image.pullPolicy | string | `"IfNotPresent"` |  |
+| image.repository | string | `"timberio/vector"` |  |
+| image.tag | string | `""` |  |
+| livenessProbe | object | `{}` | Override default liveness probe settings |
+| nodeSelector | object | `{}` | Allow Vector to be scheduled on selected nodes |
+| persistence.accessModes | list | `["ReadWriteOnce"]` | Specifies the accessModes for PersistentVolumeClaims |
+| persistence.enabled | bool | `false` | If true, create and use PersistentVolumeClaims |
+| persistence.existingClaim | string | `""` | Name of an existing PersistentVolumeClaim to use |
+| persistence.finalizers | list | `["kubernetes.io/pvc-protection"]` | Specifies the finalizers of PersistentVolumeClaims |
+| persistence.hostPath.path | string | `"/var/lib/vector"` | Override path used for hostPath persistence |
+| persistence.selectors | object | `{}` | Specifies the selectors for PersistentVolumeClaims |
+| persistence.size | string | `"10Gi"` | Specifies the size of PersistentVolumeClaims |
+| podSecurityContext | object | `{}` | Allows you to overwrite the default PodSecurityContext on the Daemonset or StatefulSet |
+| rbac.create | bool | `true` | If true, create and use RBAC resources |
+| readinessProbe | object | `{}` | Override default readiness probe settings, if customConfig is used require customConfig.api.enabled true |
+| replicas | int | `1` | Set the number of pods to create |
+| resources | object | `{}` | Vector resource requests and limits. |
+| role | string | `"Aggregator"` | Role for this deployment (possible values: Agent, Aggregator, Stateless-Aggregator) |
+| securityContext | object | `{}` | Specify securityContext on the vector container |
+| service.enabled | bool | `true` | If true, create and use a Service resource |
+| serviceAccount.create | bool | `true` | If true, create ServiceAccount |
+| serviceAccount.name | string | `nil` | The name of the ServiceAccount to use. |
+| tolerations | list | `[]` | Allow Vector to schedule on tainted nodes (requires Kubernetes >= 1.6) |
+| updateStrategy | object | `{}` | Customize the updateStrategy used to replace Vector Pods |
diff --git a/charts/vector/README.md.gotmpl b/charts/vector/README.md.gotmpl
@@ -0,0 +1,68 @@
+# Vector
+
+{{ template "chart.versionBadge" . }}{{ template "chart.typeBadge" . }}{{ template "chart.appVersionBadge" . }}
+
+[Vector](https://vector.dev/) is a high-performance, end-to-end observability data pipeline that puts you in control of your observability data. Collect, transform, and route all your logs, metrics, and traces to any vendors you want today and any other vendors you may want tomorrow. Vector enables dramatic cost reduction, novel data enrichment, and data security where you need it, not where is most convenient for your vendors.
+
+## How to use Vector Helm repository
+
+You need to add this repository to your Helm repositories:
+
+```
+helm repo add vector https://helm.vector.dev
+helm repo update
+```
+
+{{ template "chart.requirementsSection" . }}
+
+## Quick start
+
+By default, Vector runs in the Aggregator role. It can alternatively run as a DaemonSet for the Agent role.
+
+### Installing the Vector chart
+
+To install the chart with the release name `<RELEASE_NAME>` run:
+
+```bash
+helm install --name <RELEASE_NAME> \
+  vector/vector
+```
+
+### Upgrading
+
+TODO
+
+### Uninstalling the Chart
+
+To uninstall/delete the `<RELEASE_NAME>` deployment:
+
+```bash
+helm delete <RELEASE_NAME> --purge
+```
+
+The command removes all the Kubernetes components associated with the chart and deletes the release.
+
+## Configuration
+
+1. Using our [`default-values.yaml`](values.yaml) configuration file as a reference, create your own `values.yaml`.
+1. Upgrade the Vector chart with your new `values.yaml` file:
+
+```bash
+helm upgrade -f values.yaml <RELEASE_NAME> vector/vector
+```
+
+**Vector recommends that your `values.yaml` only contain values that need to be overridden, as it allows a smoother experience when upgrading chart versions.**
+
+See the [All configuration options](#all-configuration-options) section to discover all possibilities offered by the Vector chart.
+
+## All configuration options
+
+The following table lists the configurable parameters of the Vector chart and their default values. Specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. For example,
+
+```bash
+helm install --name <RELEASE_NAME> \
+  --set role=Agent \
+  vector/vector
+```
+
+{{ template "chart.valuesSection" . }}