terraform-aws-eks-clickhouse

Terraform module for creating EKS clusters optimized for ClickHouse® with EBS and autoscaling. It includes the Altinity Kubernetes Operator for ClickHouse and a fully working ClickHouse cluster.

Prerequisites

• terraform (recommended >= v1.5)
• kubectl
• aws-cli

Usage

Create an EKS Cluster with the Altinity Kubernetes Operator for ClickHouse and ClickHouse Cluster

Paste the following Terraform sample module into a tf file (main.tf) in a new directory. Adjust properties as desired. The sample module will create a Node Pool for each combination of instance type and subnet. For example, if you have 3 subnets and 2 instance types, this module will create 6 different Node Pools.

locals {
  region = "us-east-1"
}

provider "aws" {
  # https://registry.terraform.io/providers/hashicorp/aws/latest/docs
  region = local.region
}

module "eks_clickhouse" {
  source  = "github.com/Altinity/terraform-aws-eks-clickhouse"

  install_clickhouse_operator = true
  install_clickhouse_cluster  = true

  # Set to true if you want to use a public load balancer (and expose ports to the public Internet)
  clickhouse_cluster_enable_loadbalancer = false

  eks_cluster_name = "clickhouse-cluster"
  eks_region       = local.region
  eks_cidr         = "10.0.0.0/16"

  eks_availability_zones = [
    "${local.region}a",
    "${local.region}b",
    "${local.region}c"
  ]
  eks_private_cidr = [
    "10.0.1.0/24",
    "10.0.2.0/24",
    "10.0.3.0/24"
  ]
  eks_public_cidr = [
    "10.0.101.0/24",
    "10.0.102.0/24",
    "10.0.103.0/24"
  ]

  eks_node_pools = [
    {
      name          = "clickhouse"
      instance_type = "m6i.large"
      desired_size  = 0
      max_size      = 10
      min_size      = 0
      zones         = ["${local.region}a", "${local.region}b", "${local.region}c"]
    },
    {
      name          = "system"
      instance_type = "t3.large"
      desired_size  = 1
      max_size      = 10
      min_size      = 0
      zones         = ["${local.region}a"]
    }
  ]

  eks_tags = {
    CreatedBy = "mr-robot"
  }
}

⚠️ The instance type of eks_node_pools at index 0 will be used for setting up the clickhouse cluster replicas.

Run Terraform to create the cluster

Execute commands to initialize and apply the Terraform module. It will create an EKS cluster and install a ClickHouse sample database.

terraform init
terraform apply

Setting up the EKS cluster and sample database takes from 10 to 30 minutes depending on the load in your cluster and availability of resources.

Access your ClickHouse database

Update your kubeconfig with the credentials of your new EKS Kubernetes cluster.

aws eks update-kubeconfig --region us-east-1 --name clickhouse-cluster

Connect to your ClickHouse server using kubectl exec.

kubectl exec -it chi-eks-dev-0-0-0 -n clickhouse -- clickhouse-client

Run Terraform to remove the cluster

After use you can destroy the EKS cluster. First, delete any ClickHouse clusters you have created.

kubectl delete chi --all --all-namespaces

Then, run terraform destroy to remove the EKS cluster and any cloud resources.

terraform destroy

Docs

• Terraform Registry
• Architecture

Issues

If a terraform operation does not complete, try running it again. If the problem persists, please file an issue.

More Information and Commercial Support

Altinity is the maintainer of this project. Altinity offers a range of services related to ClickHouse and analytic applications on Kubernetes.

• Official website - Get a high level overview of Altinity and our offerings.
• Altinity.Cloud - Run ClickHouse in our cloud or yours.
• Altinity Support - Get Enterprise-class support for ClickHouse.
• Slack - Talk directly with ClickHouse users and Altinity devs.
• Contact us - Contact Altinity with your questions or issues.

Legal

All code, unless specified otherwise, is licensed under the Apache-2.0 license. Copyright (c) 2024 Altinity, Inc.