Skip to content

Latest commit

 

History

History
108 lines (80 loc) · 5.87 KB

README.md

File metadata and controls

108 lines (80 loc) · 5.87 KB
Raw

AWS Msk Kafka Cluster

GitHub tag (latest by date) Msk Kafka Cluster

Terraform module which creates Msk Kafka Cluster on AWS.

These types of resources are supported:

Features

This module create a fully featured Msk Kafka Cluster on Aws. You could configure monitoring, encryption, server options, etc...

Usage

module "kafka" {
  source = "angelabad/msk-cluster"

  cluster_name    = "kafka"
  instance_type   = "kafka.t3.small"
  number_of_nodes = 2
  client_subnets  = ["subnet-0ab97cbe1bd1406c2", "subnet-0d6cbf60360dbac64"]
  kafka_version   = "2.4.1"

  extra_security_groups = ["sg-019fc0f7d26f6c70f"]

  enhanced_monitoring = "PER_BROKER"

  s3_logs_bucket = aws_s3_bucket.logs.id
  s3_logs_prefix = "msklogs"

  prometheus_jmx_exporter  = true
  prometheus_node_exporter = true

  server_properties = {
    "auto.create.topics.enable"  = "true"
    "default.replication.factor" = "2"
  }

  encryption_in_transit_client_broker = "TLS"

  tags = {
    Owner       = "user"
    Environment = "dev"
  }
}

Providers

Name Version
aws >= 2.55
random >= 2.1

Inputs

Name Description Type Default Required
client_subnets A list of subnets to connect to in client VPC list(string) n/a yes
cloudwatch_logs_group Name of the Cloudwatch Log Group to deliver logs to. string "" no
cluster_name Name of the MSK cluster. string n/a yes
encryption_at_rest_kms_key_arn You may specify a KMS key short ID or ARN (it will always output an ARN) to use for encrypting your data at rest. If no key is specified, an AWS managed KMS ('aws/msk' managed service) key will be used for encrypting the data at rest. string "" no
encryption_in_transit_client_broker Encryption setting for data in transit between clients and brokers. Valid values: TLS, TLS_PLAINTEXT, and PLAINTEXT. Default value is TLS_PLAINTEXT. string "TLS_PLAINTEXT" no
encryption_in_transit_in_cluster Whether data communication among broker nodes is encrypted. Default value: true. bool true no
enhanced_monitoring Specify the desired enhanced MSK CloudWatch monitoring level to one of three monitoring levels: DEFAULT, PER_BROKER, or PER_TOPIC_PER_BROKER. See Monitoring Amazon MSK with Amazon CloudWatch. string "DEFAULT" no
extra_security_groups A list of extra security groups to associate with the elastic network interfaces to control who can communicate with the cluster. list(string) [] no
firehose_logs_delivery_stream Name of the Kinesis Data Firehose delivery stream to deliver logs to. string "" no
instance_type Specify the instance type to use for the kafka brokers. e.g. kafka.m5.large. string n/a yes
kafka_version Specify the desired Kafka software version. string n/a yes
number_of_nodes The desired total number of broker nodes in the kafka cluster. It must be a multiple of the number of specified client subnets. number n/a yes
prometheus_jmx_exporter Indicates whether you want to enable or disable the JMX Exporter. bool false no
prometheus_node_exporter Indicates whether you want to enable or disable the Node Exporter. bool false no
s3_logs_bucket Name of the S3 bucket to deliver logs to. string "" no
s3_logs_prefix Prefix to append to the folder name. string "" no
server_properties A map of the contents of the server.properties file. Supported properties are documented in the MSK Developer Guide. map(string) {} no
tags A mapping of tags to assign to the resource. map(string) {} no
volume_size The size in GiB of the EBS volume for the data drive on each broker node. number 1000 no

Outputs

Name Description
arn Amazon Resource Name (ARN) of the MSK cluster.
bootstrap_brokers A comma separated list of one or more hostname:port pairs of kafka brokers suitable to boostrap connectivity to the kafka cluster. Only contains value if client_broker encryption in transit is set o PLAINTEXT or TLS_PLAINTEXT.
bootstrap_brokers_tls A comma separated list of one or more DNS names (or IPs) and TLS port pairs kafka brokers suitable to boostrap connectivity to the kafka cluster. Only contains value if client_broker encryption in transit is set to TLS_PLAINTEXT or TLS.
current_version Current version of the MSK Cluster used for updates, e.g. K13V1IB3VIYZZH
default_security_group Msk cluster default security group id.
encryption_at_rest_kms_key_arn The ARN of the KMS key used for encryption at rest of the broker data volumes.
zookeeper_connect_string A comma separated list of one or more hostname:port pairs to use to connect to the Apache Zookeeper cluster.

NOTE: The API does not support deleting MSK configurations.

Authors

Module managed by Angel Abad

License

Apache 2 Licensed. See LICENSE for full details