This guide explains how to set up monitoring for KUDO Cassandra.
The KUDO Cassandra operator can export metrics to Prometheus. It achieves this using a Prometheus exporter based on the criteo/cassandra_exporter.
When the PROMETHEUS_EXPORTER_ENABLED
parameter is set to true
:
- A
prometheus-exporter
container will run in the same pod as every Cassandranode
container. It will listen for connections onPROMETHEUS_EXPORTER_PORT
, which is set to7200
by default. - A
prometheus-exporter-port
will be added to the KUDO Cassandra operator Service. - A ServiceMonitor will be created to make Prometheus poll that port for metrics.
- KUDO Cassandra instance running
- Prometheus operator and Grafana set up in the cluster. The kube-prometheus project provides both of them.
- KUDO CLI installed.
The examples below assume that the instance and namespace names are stored in the following shell variables. With this assumptions met, you should be able to copy-paste the commands easily.
instance_name=cassandra
namespace_name=default
This parameter is false
by default, so you need to enable it explicitly.
You can check the value of the parameter on a running instance with a command like:
kubectl get instance --template '{{.spec.parameters.PROMETHEUS_EXPORTER_ENABLED}}{{"\n"}} $instance_name -n $namespace_name'
An output of true
means that the exporter is enabled.
Any other output means that the exporter is disabled. In that case you need
to enable it with a command such as the following. If you need customization,
see other parameters that start with PROMETHEUS_EXPORTER_
.
kubectl kudo update -p PROMETHEUS_EXPORTER_ENABLED=true --instance $instance_name -n $namespace_name
Expected output:
Instance cassandra was updated.
A sample grafana dashboard is provided in the monitoring directory.
How you access the Grafana UI depends on how it was installed. Upon accessing
the /dashboard/import
URI you will be able to upload or copy-paste the
cassandra.json
file:
Once done, you will be able to see various Cassandra metrics in the dashboard:
Prometheus
resource. The serviceMonitorNamespaceSelector
and
serviceMonitorSelector
attributes on that resource need to be configured to
match the
labels on the ServiceMonitor
resource
created by the KUDO Cassandra operator.
The Prometheus exporter container that is run alongside each Cassandra node requires 1 CPU and 512MiB memory each.
To use the custom prometheus exporter configuration, we need to create a configmap with the properties we want to override.
Example custom configuration:
apiVersion: v1
kind: ConfigMap
metadata:
name: custom-exporter-configuration
data:
config.yml: |
maxScrapFrequencyInSec:
2000:
- .*:totaldiskspaceused:.*
Create the ConfigMap in the namespace we will have the KUDO Cassandra cluster
$ kubectl create -f custom-exporter-configuration.yaml -n $namespace_name
configmap/custom-exporter-configuration created
Enable the exporter
kubectl kudo update \
-p PROMETHEUS_EXPORTER_ENABLED=true \
-p PROMETHEUS_EXPORTER_CUSTOM_CONFIG_CM_NAME=custom-exporter-configuration \
--instance $instance_name -n $namespace_name
- host
- listenAddress
- listenPort
- user
- password
- ssl