Skip to content

The Kubernetes Controller to dynamically inject Snowflake ID as environment variables on annotated pods at runtime

Notifications You must be signed in to change notification settings

NaturalSelectionLabs/snowflake-id

 
 

Repository files navigation

snowflake_id

Snowflake ID is an algorithm created by Twitter for generating unique identifiers under distributed systems. This project is meant to create these IDs for applications hosted in the Kubernetes environment. With the help of AdmissionWebhook, the customized snowflake-id controller can intercept the pod creation/deletion events from Kubernetes API server and verify if the pod is annotated with the label snowflake-id.io/enabled with true value. If so, the controller will modify the pod spec for you, attaching generated values, SNOWFLAKE_DATA_CENTER_ID and SNOWFLAKE_WORKER_ID, to the environment fields, and deploy the pod to the designated node.

System Design

Terminology Redefinition

10 bits represent a machine ID consisting of 5 bits for worker nodes and 5 bits for pods

  • Data Center ID -> worker node
  • Worker ID -> pod

Limitation factor

  1. A single microservice can only be deployed to the utmost 32 worker nodes
  2. A single microservice can only be deployed to a single worker node for the utmost 32 pods
  3. A single microservice can have 1024 pods (32*32)

snowflake-id

Local development

Prerequisite

Download Software

  • kind - a local Kubernetes cluster
  • Skaffold - Local Kubernetes Development.

Setup kind & Skaffold configuration files

  1. clone current project
  2. cd snowflake-id
  3. create the following files
    • # kind.yaml
      ---
      kind: Cluster
      apiVersion: kind.x-k8s.io/v1alpha4
      name: snowflake-id-cluster-test
      nodes:
        - role: control-plane
          image: kindest/node:v1.21.1@sha256:69860bda5563ac81e3c0057d654b5253219618a22ec3a346306239bba8cfa1a6
        - role: worker
          image: kindest/node:v1.21.1@sha256:69860bda5563ac81e3c0057d654b5253219618a22ec3a346306239bba8cfa1a6
        - role: worker
          image: kindest/node:v1.21.1@sha256:69860bda5563ac81e3c0057d654b5253219618a22ec3a346306239bba8cfa1a6
        - role: worker
          image: kindest/node:v1.21.1@sha256:69860bda5563ac81e3c0057d654b5253219618a22ec3a346306239bba8cfa1a6
      
    • # skaffold.yaml
      apiVersion: skaffold/v2beta27
      kind: Config
      metadata:
        name: local-cluster
      build:
        artifacts:
        - image: snowflake_id
          # An alpha feature https://skaffold.dev/docs/pipeline-stages/lifecycle-hooks/
          #
          # WARNING - please create a target file (in this case should be `webhook`) before executing skaffold
          hooks:
            before:
              - command: ["sh", "-c", "./compile.sh"]
          context: .
          docker:
            dockerfile: Dockerfile
      deploy:
        helm:
          releases:
          - name: "snowflake-id-chart"
            artifactOverrides:
              image: snowflake_id # no tag present!
            imageStrategy:
              helm: {}
            chartPath: # the path to your local helm project
            valuesFiles: # the value file of your local helm project
      
  4. skaffold dev

Happy coding!

About

The Kubernetes Controller to dynamically inject Snowflake ID as environment variables on annotated pods at runtime

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Go 96.8%
  • Dockerfile 3.2%