Scaling namespace storage

Scaling namespace storage (vertical scaling) is a bit complex for operator. Operator uses k8s StatefulSet for deploying Aerospike-cluster. StatefulSet uses PersistentVolumeClaim for providing storage. Currently this PersistentVolumeClaim can not be updated. Hence operator can not provide a simple solution for vertical scaling. But we can exploit the rack-awareness feature of operator for vertical scaling.

For this example assume that cluster is deployed using given a named aerospike-cluster.yaml.

apiVersion: aerospike.com/v1alpha1
kind: AerospikeCluster
metadata:
  name: aerocluster
  namespace: aerospike

spec:
  size: 2
  build: aerospike/aerospike-server-enterprise:4.7.0.10

  rackConfig:
    namespaces:
      - test
    racks: 
      - id: 1
        zone: us-central1-b
        storage:
          volumes:
            - path: /dev/sdf
              storageClass: ssd
              volumeMode: block
              sizeInGB: 5
  
  aerospikeConfig:
    service:
      feature-key-file: /etc/aerospike/secret/features.conf
    security:
      enable-security: true
    namespace:
      - name: test
        memory-size: 6000000000
        replication-factor: 2
        storage-engine:
          device:
            - /dev/sdf
.
.
.

Now if we want to resize /dev/sdf for namespace test then we have to create a new rack inside rackConfig with updated storage config and remove the old rack. New rack can be created in same physical rack using exsiting zone/region if there is enough space to hold new storage and old storage together.

Update the rackConfig section

apiVersion: aerospike.com/v1alpha1
kind: AerospikeCluster
metadata:
  name: aerocluster
  namespace: aerospike

spec:
  size: 2
  build: aerospike/aerospike-server-enterprise:4.7.0.10

  rackConfig:
    namespaces:
      - test
    racks: 
      # Added new rack with id: 2. Old rack with id: 1 is removed
      - id: 2
        zone: us-central1-b
        storage:
          volumes:
            - path: /dev/sdf
              storageClass: ssd
              volumeMode: block
              sizeInGB: 8
  
  aerospikeConfig:
    service:
      feature-key-file: /etc/aerospike/secret/features.conf
    security:
      enable-security: true
    namespace:
      - name: test
        memory-size: 10000000000
        replication-factor: 2
        storage-engine:
          device:
            - /dev/sdf
.
.
.

Apply the change

$ kubectl apply -f aerospike-cluster.yaml

This will create a new rack with id: 2 and updated storage config. Old data will be migrated to new rack. Old rack will be removed gracefully.

Check the pods

$ kubectl get pods -n aerospike
NAME                READY   STATUS          RESTARTS   AGE
aerocluster-2-0     1/1     Running         0          3m6s
aerocluster-2-1     1/1     Running         0          3m6s
aerocluster-1-1     1/1     Terminating     0          30s