Skip to content

Commit

Permalink
Merge pull request #95 from survivant/develop
Browse files Browse the repository at this point in the history 
add custom dashboard and rules
  • Loading branch information
@survivant
survivant authored Oct 27, 2021
2 parents 56712ac + ad30928 commit 1cf3150
Showing 1 changed file with 217 additions and 7 deletions.
224 changes: 217 additions & 7 deletions docs/extends-stack-with-umbrella-chart.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ An umbrella chart is a way of encapsulate multiple complex charts with one chart
>Many of the charts in the CNCF Artifact Hub are "building blocks" for creating more advanced applications. But charts may be used to create instances of large-scale applications. In such cases, a single umbrella chart may have multiple subcharts, each of which functions as a piece of the whole.
>The current best practice for composing a complex application from discrete parts is to create a top-level umbrella chart that exposes the global configurations, and then use the charts/ subdirectory to embed each of the components.
Source: https://helm.sh/docs/howto/charts_tips_and_tricks/#complex-charts-with-many-dependencies
Source: https://helm.sh/docs/howto/charts_tips_and_tricks/#complex-charts-with-many-dependencies

Using an umbrella chart will allow you customize OpenEBS monitoring stack settings without modifying that chart content. It's also easier to update a dependency.

Expand All @@ -33,7 +33,7 @@ To create your our umbrella chart, you will have a create a folder structure lik

Here a example that will enable ingress and storage for OpenEBS monitoring stack.

In the file **Chart.yaml**, you will enter your umbrella chart information with a dependency to OpenEBS monitoring stack. The property name : `condition` is optional. That property will allow you to install that dependency or not depending of the value in `values.yaml`.
In the file **Chart.yaml**, you will enter your umbrella chart information with a dependency to OpenEBS monitoring stack. The property name : `condition` is optional. That property will allow you to install that dependency or not depending of the value in `values.yaml`.

#### Chart.yaml
````yaml
Expand All @@ -51,9 +51,9 @@ dependencies:
condition: openebs-monitoring.install
````

In the file **values.yaml**, you will define your configuration for the monitoring stack.
In the file **values.yaml**, you will define your configuration for the monitoring stack.

If you defined a **dependency condition** in **Chart.yaml**, you will have to define the value in that condition. In this example, the condition name is `openebs-monitoring.install`.
If you defined a **dependency condition** in **Chart.yaml**, you will have to define the value in that condition. In this example, the condition name is `openebs-monitoring.install`.

if you provide only the condition value, you will deploy OpenEBS monitoring stack with the default values provided in there chart.

Expand All @@ -67,7 +67,7 @@ openebs-monitoring:

To use storage for the monitoring stack, you will have to create the storage class before using it. In the following example, I use a storage class named: `sc-metrics` which already exists in my cluster.

Similarly you can also customize other parameters. For instance, if domain name has to be used to access the user interfaces, the `ingress` section needs to be updated. In the following example I use the value: `dev.dynamic.cluster109.local`.
Similarly you can also customize other parameters. For instance, if domain name has to be used to access the user interfaces, the `ingress` section needs to be updated. In the following example I use the value: `dev.dynamic.cluster109.local`.

The applications will be accessible from :
- http://dev.dynamic.cluster109.local/prometheus
Expand Down Expand Up @@ -179,10 +179,220 @@ openebs-monitoring:
install: true
````

# Add your Grafana Dashboards and Prometheus rules

OpenEBS monitoring stack come with Grafana dashboards and Prometheus rules, but it's possible to add your own by extending the monitoring stack.

In the monitoring stack, there is a folder named `dashboards` that contains Grafana dashboards and the folder `rules` contains Prometheus rules. To extends the stack, we will have the folder structure based on the structure within the monitoring stack.

Our umbrella chart folder structure will look like that :

| Folder/File | Description |
| ------ | ------ |
| charts | Contains your chart dependencies (auto-generated with helm) |
| dashboards | Contains all your Grafana dashboards (json format) |
| rules | Contains all your Prometheus rules (json or yaml format) |
| templates | helm templates to include your dashboards and rules into OpenEBS monitoring stack|
| Chart.yaml | Contains your chart identification with a dependency to OpenEBS monitoring stack |
| values.yaml | Contains all your settings values |

Here a example of umbrella chart using external Grafana dashboards and Prometheus rules. You can separate your files with sub folders to simplify and regroup them. The helm templates will check within the sub folders for files to import.

````
📦umbrella
┣ 📂charts
┃ ┗ 📜openebs-monitoring-0.4.9.tgz
┣ 📂dashboards
┃ ┣ 📂custom
┃ ┃ ┣ 📜k8s-persistence-volumes.json
┃ ┃ ┗ 📜node-exporter-full.json
┃ ┗ 📂kafka
┃ ┃ ┣ 📜confluent-open-source-grafana-dashboard.json
┣ 📂rules
┃ ┗ 📂custom
┃ ┃ ┗ 📜instance-down.json
┣ 📂templates
┃ ┣ 📜dashboards-json-configmap.yaml
┃ ┣ 📜prometheusRules.yaml
┃ ┗ 📜_helpers.tpl
┣ 📜Chart.lock
┣ 📜Chart.yaml
┗ 📜values.yaml
````
### Helm templates to import your files into OpenEBS monitoring stack
Here the content of the files into templates folder. Those files are based on those within the monitoring stack. You can customize it to fit your needs or use it as is.
| File | Description |
| ------ | ------ |
| _helpers.tpl | Helm template helper |
| dashboard-json-configmap.yaml | Contains the logic to add Grafana dashboards to OpenEBS monitoring stack |
| prometheusRules.yaml | Contains the logic to add Prometheus rules to OpenEBS monitoring stack |
#### _helpers.tpl
````yaml
{{/* vim: set filetype=mustache: */}}
{{/*
Expand the name of the chart.
*/}}
{{- define "openebs-monitoring.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 50 | trimSuffix "-" }}
{{- end }}
{{/*
Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
If release name contains chart name it will be used as a full name.
*/}}
{{- define "openebs-monitoring.fullname" -}}
{{- if .Values.fullnameOverride }}
{{- .Values.fullnameOverride | trunc 26 | trimSuffix "-" }}
{{- else }}
{{- $name := default .Chart.Name .Values.nameOverride }}
{{- if contains $name .Release.Name }}
{{- .Release.Name | trunc 26 | trimSuffix "-" }}
{{- else }}
{{- printf "%s-%s" .Release.Name $name | trunc 26 | trimSuffix "-" }}
{{- end }}
{{- end }}
{{- end }}
{{/*
Allow the release namespace to be overridden for multi-namespace deployments in combined charts
*/}}
{{- define "openebs-monitoring.namespace" -}}
{{- if .Values.namespaceOverride -}}
{{- .Values.namespaceOverride -}}
{{- else -}}
{{- .Release.Namespace -}}
{{- end -}}
{{- end -}}
{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "openebs-monitoring.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
Common labels
*/}}
{{- define "openebs-monitoring.labels" -}}
helm.sh/chart: {{ include "openebs-monitoring.chart" . }}
{{ include "openebs-monitoring.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}
{{/*
Selector labels
*/}}
{{- define "openebs-monitoring.selectorLabels" -}}
app.kubernetes.io/name: {{ include "openebs-monitoring.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
release: {{ $.Release.Name | quote }}
{{- end }}
{{/*
Template to support multiple levels of sub-charts
Call a template from the context of a subchart.
Usage:
{{ include "call-nested" (list . "<subchart_name>" "<subchart_template_name>") }}
*/}}
{{- define "call-nested" }}
{{- $dot := index . 0 }}
{{- $subchart := index . 1 | splitList "." }}
{{- $template := index . 2 }}
{{- $values := $dot.Values }}
{{- range $subchart }}
{{- $values = index $values . }}
{{- end }}
{{- include $template (dict "Chart" (dict "Name" (last $subchart)) "Values" $values "Release" $dot.Release "Capabilities" $dot.Capabilities) }}
{{- end }}
````

#### dashboard-json-configmap.yaml
````yaml
{{- $grafanaSidecarDashboardsLabel := index .Values "openebs-monitoring" "kube-prometheus-stack" "grafana" "sidecar" "dashboards" "label" }}
{{- $files := .Files.Glob "dashboards/**.json" }}
{{- if $files }}
{{- range $fileName, $fileContents := $files }}
{{- $dashboardName := regexReplaceAll "(^.*/)(.*)\\.json$" $fileName "${2}" }}
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ lower (printf "%s-%s" (include "openebs-monitoring.fullname" $) $dashboardName) | trunc 63 | trimSuffix "-" }}
namespace: {{ template "openebs-monitoring.namespace" $ }}
labels:
{{ $grafanaSidecarDashboardsLabel }}: "1"
app: {{ template "openebs-monitoring.name" $ }}-grafana
{{ include "openebs-monitoring.labels" $ | indent 4 }}
data:
{{ $dashboardName }}.json: {{ $.Files.Get $fileName | toPrettyJson }}
---
{{- end }}
{{- end }}
````

#### prometheusRules.yaml
````yaml
{{- $files := .Files.Glob "rules/**.json" }}
{{- if $files }}
{{- range $fileName, $fileContents := $files }}
{{- $ruleName := regexReplaceAll "(^.*/)(.*)\\.json$" $fileName "${2}" }}
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
name: {{ lower (printf "%s-%s" (include "openebs-monitoring.name" $) $ruleName) | trunc 63 | trimSuffix "-" }}
namespace: {{ template "openebs-monitoring.namespace" $ }}
labels:
app: {{ template "openebs-monitoring.name" $ }}
{{ include "openebs-monitoring.labels" $ | indent 4 }}
spec:
{{ $.Files.Get $fileName | indent 2 }}
---
{{- end }}
{{- end }}
````

Here a example of a custom Prometheus rules that you can add to the monitoring stack.

#### instance-down.json
````json
{
"groups": [
{
"name": "Instances",
"rules": [
{
"alert": "InstanceDown",
"expr": "up == 0",
"for": "10s",
"labels": {
"severity": "page"
},
"annotations": {
"description": "{{ $labels.instance }} of job {{ $labels.job }} has been down for more than 1 minute.",
"summary": "Instance {{ $labels.instance }} down"
}
}
]
}
]
}
````

# How to deploy the umbrella chart

### Download the dependencies
The first step will be to download the dependencies by running this command in a terminal in the umbrella folder.
The first step will be to download the dependencies by running this command in a terminal in the umbrella folder.

````console
helm dep build
Expand Down Expand Up @@ -229,7 +439,7 @@ In a terminal in the umbrella folder run this command :
helm install [RELEASE_NAME] [REPOSITORY]/[CHART_NAME] -n [NAMESPACE] --create-namespace
```

ex :
ex :
````console
helm install monitoring chartmuseum/monitoring-stack -n monitoring --create-namespace
````

0 comments on commit 1cf3150

Please sign in to comment.