Skip to content

Latest commit

 

History

History
362 lines (271 loc) · 12.1 KB

helm.md

File metadata and controls

362 lines (271 loc) · 12.1 KB

Helm

:::{include} ../.internal/helm-crd-warning.md :::

In this example we will install Scylla stack on Kubernetes. This includes the following components:

  • Scylla Operator
  • Scylla Manager
  • Scylla

We will use Minikube K8s cluster, but this could be any K8s cluster supported by the Scylla Operator.

Prerequisites

  • Kubernetes 1.16+
  • Helm 3+

TL;DR

helm repo add scylla https://scylla-operator-charts.storage.googleapis.com/stable
helm repo update
kubectl apply -f examples/common/cert-manager.yaml 
helm install scylla-operator scylla/scylla-operator --create-namespace --namespace scylla-operator
helm install scylla-manager scylla/scylla-manager --create-namespace --namespace scylla-manager
helm install scylla scylla/scylla --create-namespace --namespace scylla

Deploy Cert Manager

This step is optional if you want to use your own certificate. If you don't have one, make sure to not disable autogeneration using Scylla Operator Helm Chart.

First deploy Cert Manager, you can either follow upsteam instructions or use following command:

kubectl apply -f examples/common/cert-manager.yaml

Once it's deployed, wait until all Cert Manager pods will enter into Running state:

kubectl wait -n cert-manager --for=condition=ready pod -l app=cert-manager --timeout=60s

Helm Chart repository

To install Scylla Helm Chart repository execute the following commands:

helm repo add scylla https://scylla-operator-charts.storage.googleapis.com/stable
helm repo update

Then you can search through repository, it should contain at least three Helm charts:

helm search repo scylla
NAME                   CHART VERSION   APP VERSION     DESCRIPTION                                       
scylla/scylla          1.0.1           v1.0.1          Scylla is a close-to-the-hardware rewrite of Ca...
scylla/scylla-manager  1.0.1           v1.0.1          Scylla Manager automates database operations.     
scylla/scylla-operator 1.0.1           v1.0.1          Scylla Operator is a Kubernetes Operator for ma...

All these charts should be installable without any need of customizing (defaults are provided). Although Helm is used for this particular reason, so lets customize them a bit.

Scylla Operator Chart

This chart is very simple, most interesting customizable fields are image, resources and webhook. All others can be looked up in Chart source in Scylla Operator repository.

image

Image allows to define which Scylla Operator image will be used. By default it downloads the image from main Docker Hub repository, using version defined in Helm Chart. You can also change pullPolicy if default one does not fullfill your needs. In Kubernetes documentation you can read more about different pull policies.

Image URL will be composed based on these fields in follwing pattern: repository/scylla-operator:tag

image:
  repository: scylladb
  pullPolicy: IfNotPresent
  tag: ""

resources

You can customize how much resources will be allocated for Operator pods via resource field:

resources:
  limits:
    cpu: 100m
    memory: 128Mi
  requests:
    cpu: 100m
    memory: 32Mi

To read more about resource specification, follow Kubernetes documentation.

webhook

Webhook field allows to decide whether you want to use autogenerated self-signed certificate using Cert Manager or whether you want to provide your own certificate.

createSelfSignedCertificate specifies whether a self-signed certificate should be created using Cert Manager certificateSecretName: name of a secret containing custom certificate.

webhook:
  createSelfSignedCertificate: true
  certificateSecretName: ""

Customization

You can customize all these fields and others by providing file containing desired values. Content of this file will overwrite default values.

You can find an example in Scylla Operator repository under examples/helm/values.operator.yaml

Installation

To deploy Scylla Operator using customized values file execute the following:

helm install scylla-operator scylla/scylla-operator --values examples/helm/values.operator.yaml --create-namespace --namespace scylla-operator

Scylla Helm Chart

Scylla Chart allows to customize and deploy Scylla cluster. By default Scylla Helm charts deploys working Scylla cluster, but of course we can customize it.

Customization

Versions of images used in the cluster can be set via scyllaImage and agentImage

scyllaImage:
  repository: scylladb/scylla
  tag: 4.3.0

agentImage:
  repository: scylladb/scylla-manager-agent
  tag: 2.2.1

A minimal Scylla cluster can be expressed as:

datacenter: us-east-1
racks:
- name: us-east-1b
  members: 2
  storage:
    capacity: 5G
  resources:
    limits:
      cpu: 1
      memory: 1Gi
    requests:
      cpu: 1
      memory: 1Gi

Above cluster will use 4.3.0 Scylla, 2.2.1 Scylla Manager Agent sidecar and will have a single rack having 2 nodes. Each node will have a single CPU and 1 GiB of memory.

For other customizable fields, please refer to ScyllaCluster CRD. CRD Rack Spec and Helm Chart Rack should have the same fields.

Installation

To deploy Scylla cluster using customzied values file execute the following command:

helm install scylla scylla/scylla --values examples/helm/values.cluster.yaml --create-namespace --namespace scylla

Scylla Operator will provision this cluster on your K8s environment.

Scylla Manager Helm Chart

Scylla Manager Chart allows to customize and deploy Scylla Manager in K8s environment. Scylla Manager consist of two applications (Scylla Manager itself and Scylla Manager Controller) and additional Scylla cluster.

To read more about Scylla Manager see Manager guide.

Scylla Manager

To set version of used Scylla Manager you can use image field:

image:
  repository: scylladb
  pullPolicy: IfNotPresent
  tag: 2.2.1

To control how many resources are allocated for Scylla Manager use resource field:

resources:
  limits:
    cpu: 500m
    memory: 500Mi
  requests:
    cpu: 500m
    memory: 500Mi

Scylla Manager Controller

Similarly Scylla Manager Controller image can be customized:

controllerImage:
  repository: scylladb
  pullPolicy: IfNotPresent
  tag: ""

And allocated resources:

controllerResources:
  limits:
    cpu: 100m
    memory: 30Mi
  requests:
    cpu: 100m
    memory: 20Mi

Scylla

To customize internal Scylla instance dedicated to Scylla Manager, see guide above customizing Scylla Helm Chart. It's definition should land as a scylla field.

Customization

All others customizable fields can be looked up in Chart source in Scylla Operator repository.

Installation

To deploy Scylla Manager using customized values file execute the following command:

helm install scylla-manager scylla/scylla-manager --values examples/helm/values.manager.yaml --create-namespace --namespace scylla-manager

Results

Scylla need some time to bootstrap all nodes, but after some time you should be ready to roll. It was simple isn't it? You can validate if everything was set up correctly by looking at the all resources created in used namespaces.

Scylla Operator:

$ kubectl -n scylla-operator get all

NAME                                   READY   STATUS    RESTARTS   AGE
pod/scylla-operator-5dbcb54f5c-vjm4m   1/1     Running   0          51s
pod/scylla-operator-5dbcb54f5c-wfjbw   1/1     Running   0          51s

NAME                              TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
service/scylla-operator-webhook   ClusterIP   10.105.207.130   <none>        443/TCP   51s

NAME                              READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/scylla-operator   2/2     2            2           51s

NAME                                         DESIRED   CURRENT   READY   AGE
replicaset.apps/scylla-operator-5dbcb54f5c   2         2         2       51s

Operator is running!

Scylla Manager:

$ kubectl -n scylla-manager get all 

NAME                                             READY   STATUS    RESTARTS   AGE
pod/scylla-manager-669db64dd-bcm4v               1/1     Running   0          89s
pod/scylla-manager-controller-844ccc56c4-drbth   1/1     Running   0          89s
pod/scylla-manager-controller-844ccc56c4-rhwqx   1/1     Running   0          89s

NAME                            TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)             AGE
service/scylla-manager          ClusterIP   10.105.231.53   <none>        80/TCP,5090/TCP     89s
service/scylla-manager-client   ClusterIP   None            <none>        9180/TCP,5090/TCP   89s

NAME                                        READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/scylla-manager              1/1     1            1           89s
deployment.apps/scylla-manager-controller   2/2     2            2           89s

NAME                                                   DESIRED   CURRENT   READY   AGE
replicaset.apps/scylla-manager-669db64dd               1         1         1       89s
replicaset.apps/scylla-manager-controller-844ccc56c4   2         2         2       89s

Good to go, ready to serve!

Scylla itself:

$ kubectl -n scylla get all        

NAME                                READY   STATUS    RESTARTS   AGE
pod/scylla-us-east-1-us-east-1b-0   2/2     Running   0          5m58s
pod/scylla-us-east-1-us-east-1b-1   2/2     Running   0          4m29s

NAME                                    TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                                                           AGE
service/scylla-client                   ClusterIP   None           <none>        9180/TCP,5090/TCP                                                 5m59s
service/scylla-us-east-1-us-east-1b-0   ClusterIP   10.43.149.92   <none>        7000/TCP,7001/TCP,7199/TCP,10001/TCP,9042/TCP,9142/TCP,9160/TCP   5m58s
service/scylla-us-east-1-us-east-1b-1   ClusterIP   10.43.49.0     <none>        7000/TCP,7001/TCP,7199/TCP,10001/TCP,9042/TCP,9142/TCP,9160/TCP   4m29s

NAME                                           READY   AGE
statefulset.apps/scylla-us-east-1-us-east-1b   2/2     5m59s

Two running nodes, exactly what we were asking for.

Monitoring

To spin up a Prometheus monitoring refer to monitoring guide.

Helm charts can create ServiceMonitors needed to observe Scylla Manager and Scylla. Both of these Helm Charts allows to specify whether you want to create a ServiceMonitor:

serviceMonitor:
  create: false

Change create to true and update your current deployment using:

helm upgrade --install scylla --namespace scylla scylla/scylla -f examples/helm/values.cluster.yaml

Helm should notice the difference, install the ServiceMonitor, and then Prometheous will be able to scrape metrics.

Upgrade via Helm

Replace <release_name> with the name of your Helm release for Scylla Operator and replace <version> with the version number you want to install:

  1. Make sure Helm chart repository is up-to-date: 
    helm repo add scylla-operator https://storage.googleapis.com/scylla-operator-charts/stable
helm repo update
  2. Update CRD resources. We recommend using --server-side flag for kubectl apply, if your version supports it. 
    tmpdir=$( mktemp -d ) \
  && helm pull scylla-operator/scylla-operator --version <version> --untar --untardir "${tmpdir}" \
  && find "${tmpdir}"/scylla-operator/crds/ -name '*.yaml' -printf '-f=%p ' \
  | xargs kubectl apply
  3. Update Scylla Operator 
    helm upgrade --version <version> <release_name> scylla-operator/scylla-operator

Cleanup

To remove these applications you can simply uninstall them using Helm CLI:

helm uninstall scylla -n scylla
helm uninstall scylla-manager -n scylla-manager
helm uninstall scylla-operator -n scylla-operator