Skip to content

canonical/k8s-dqlite

This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.

Folders and files

NameName
Last commit message
Last commit date

Latest commit

fb7b539 · Jan 13, 2025
Jan 13, 2025
Jan 13, 2025
Jan 13, 2025
Nov 15, 2024
Jan 13, 2025
Jan 13, 2025
May 14, 2021
Nov 14, 2024
Feb 27, 2024
May 14, 2021
Nov 15, 2024
Sep 27, 2024
Nov 21, 2024
Jan 13, 2025
Jan 13, 2025
Aug 30, 2023

Repository files navigation

OpenSSF Scorecard

K8s-dqlite: Dqlite Backend for Kubernetes

This project is a drop in replacement for etcd on Kubernetes.

If you're looking for a lightweight, distributed, and in-memory database to replace etcd in your K8s cluster, Dqlite might be the solution for you.

Key Features

  • Distributed SQlite: Uses Dqlite as the backend
  • In-memory: Efficient and fast
  • High Availability: Built-in high availabilty with Raft
  • Minimal operations: Designed to be almost zero ops

Building

k8s-dqlite links with Dqlite. To build the project, you can create static and dynamic binaries:

Static Build (Recommended)

make static
./bin/static/k8s-dqlite --help

Dynamic Build

If you prefer dynamic binaries (which require Dqlite shared libraries and dependencies to be present during runtime), use the following commands:

make dynamic
./bin/dynamic/k8s-dqlite --help

Installing

  1. Prepare a directory containing the necessary configuration files:
  • init.yaml with Address: <host_IP>:<port_used_by_dqlite>
  • cluster.crt and cluster.key pair.
  1. Use the following script to generate the required files:
mkdir -p /var/data/
IP="127.0.0.1"
PORT="29001"
DNS=$(/bin/hostname)

# Create init.yaml with the Dqlite listening address
echo "Address: $IP:$PORT" > /var/data/init.yaml
mkdir -p /var/tmp/

# Find the csr-dqlite.conf.template in the project
cp /config/csr-dqlite.conf.template /var/tmp/csr-dqlite.conf
sed -i 's/HOSTNAME/'"${DNS}"'/g' /var/tmp/csr-dqlite.conf
sed -i 's/HOSTIP/'"${IP}"'/g' /var/tmp/csr-dqlite.conf
openssl req -x509 -newkey rsa:4096 -sha256 -days 3650 -nodes -keyout /var/data/cluster.key -out ${SNAP_DATA}/var/data/cluster.crt -subj "/CN=k8s" -config /var/tmp/csr-dqlite.conf -extensions v3_ext
chmod -R o-rwX /var/data
  1. Start k8s-dqlite by running the following command:
k8s-dqlite --storage-dir=/var/data/

By default, k8s-dqlite listens for connections from the Kubernetes API server at tcp://127.0.0.1:12379. To change the address, use the --listen option.

Configuring the API server

To point the API server to k8s-dqlite use the following arguments:

# Endpoint k8s-dqlite listens at
--etcd-servers="http://<IP>:<PORT>"
# By default --etcd-servers="http://127.0.0.1:12379"

# Point to the cluster key pair
--etcd-certfile=/data/cluster.crt
--etcd-keyfile=/var/data/cluster.key

# If you are using the snap comment in these lines
#--etcd-certfile=/var/snap/k8s-dqlite/current/var/data/cluster.crt
#--etcd-keyfile=/var/snap/k8s-dqlite/current/var/data/cluster.key

Highly Available Dqlite

K8s-dqlite supports high availability by using the Raft protocol to achieve consensus through an elected leader. Below are the steps to set up a highly available Dqlite cluster with three nodes.

Steps

  1. On the main node, verify that k8s-dqlite is listening on the correct IP address and port. You can check this by running:
sudo ss -tuln | grep <the ip of the main node>:29001

If k8s-dqlite is listening correctly, you should see an output indicating that it's bound to the node's IP and port.

  1. On each joining node, prepare the environment:
    • Backup the Dqlite data directory (/var/data).
    • Remove the existing the Dqlite data directory.
  2. Copy the cluster.crt and cluster.key from the main node and place it into the joining node's /var/data.
  3. Create the init.yaml file with the following content
Cluster:
- <the ip of the main node>:29001
Address: <ip of the joining node>:29001
  1. Start the k8s-dqlite process on the joining node:
k8s-dqlite --storage-dir=/var/data/
  1. On the main node, verify that the joining node is connected to the Dqlite cluster.
dqlite -s file:///var/data/cluster.yaml -c /var/data/cluster.crt -k /var/data/cluster.key -f json k8s .cluster
  1. Repeat steps 2–6 for any additional nodes you wish to join to the cluster.