Skip to content

Latest commit

 

History

History
128 lines (100 loc) · 5.11 KB

proposal-003-extending-use-of-config-file.md

File metadata and controls

128 lines (100 loc) · 5.11 KB
Raw

Design Proposal #003: Extending Use of --config-file

STATUS: This proposal is a final state, and we expect minimal additional refinements. If any non-trivial changes are needed to functionality defined here, in particular the user experience, those changes should be suggested via a PR to this proposal document. Any other changes to the text of the proposal or technical corrections are also very welcome.

Support for config files had been added in eksctl 0.1.17 as an experimental alpha feature, it has become popular amongst users. Users currently can specify ClusterConfig object in JSON or YAML, which allows them to set more fields then they currently can with CLI flags. They can also specify several nodegroups, and create a cluster in one go. Besides many missing features in v1alpha4 incarnation (where there is little of management functionality beyond initial cluster creation), one of commonly requested features is management of nodegroups through config files.

This proposal aims to define how existing ClusterConfig object can be enhanced to support new type of objects that allow users to define and manage nodegroups via a separate step.

Let's consider a simple config:

# cluster.yaml
---
apiVersion: eksctl.io/v1alpha4
kind: ClusterConfig

metadata:
  name: cluster-5
  region: eu-north-1

nodeGroups:
  - name: ng1-public
    instanceType: m5.xlarge
    desiredCapacity: 4
  - name: ng2-private
    instanceType: m5.large
    desiredCapacity: 10
    privateNetworking: true

There are two nodegroups defined inline. When this file is passed to eksctl create cluster --config-file=cluster.yaml the cluster will be created with the nodegroups as defined.

To create additional nodegroups for this cluster, we will enhance the eksctl create nodegroup cli to selectively, using glob patterns, pull nodegroups from this configuration file. This enables the entire cluster definition to live in a single file.

For example, if we add 2 nodegroups to our original cluster config:

# cluster.yaml
---
apiVersion: eksctl.io/v1alpha4
kind: ClusterConfig

metadata:
  name: cluster-5
  region: eu-north-1

nodeGroups:
  - name: ng1-public
    instanceType: m5.xlarge
    desiredCapacity: 4
  - name: ng2-private
    instanceType: m5.large
    desiredCapacity: 10
    privateNetworking: true
  - name: ng3-dev
    instanceType: m4.large
    desiredCapacity: 2
    privateNetworking: true
  - name: ng2-test
    instanceType: m5.large
    desiredCapacity: 4
    privateNetworking: true

eksctl create nodegroup will be updated to support glob pattern matching for nodegroup names contained within the cluster config. Therefore, a user can create these additional nodegroups using eksctl create nodegroup --config-file=cluster.yaml --include="*dev,*test". eksctl will create two nodegroups in cluster named cluster-5.

Here are some additional examples:

eksctl create ng --config-file=cluster.yaml ## all nodegroups will be created
eksctl create ng --config-file=cluster.yaml --include=ng1-public  ## only one nodegroup will be created
eksctl create ng --config-file=cluster.yaml --include=ng1-public,ng2-private ## both nodegroups will be created

The eksctl create cluster cli will be updated to optionally ignore all nodegroups in the cluster config. eksctl-io#555

By using a single configuration file, users can keep their cluster definitions together and most importantly checked into source control.

Possible future enhancements

  • Update eksctl create cluster to allow the user to create selected nodegroups defined in the cluster config

    eksctl create cluster --config-file=cluster.yaml --include='.*-private' ## only nodegroups matching glob pattern will be created

  • Support storing nodegroups within their own configuration files.

For example:

# extra-nodegroup.yaml
---
apiVersion: eksctl.io/v1alpha4
kind: NodeGroupConfig

metadata:
  cluster: cluster-5
  region: eu-north-1
  name: ng3-extra

spec:
  instanceType: c4.xlarge
  minSize: 10
  maxSize: 20
  privateNetworking: true

Two ways that these config files can be used to create cluster:

  • eksctl create cluster --config-file=cluster.yaml && eksctl create ng --config-file=extra-nodegroup.yaml
  • cat cluster.yaml extra-nodegroup.yaml | eksctl create cluster --config-file -

Questions

  • Is it appropriate to also allow referencing nodegroup that is defined externally within ClusterConfig?
    • Yes. That is the initial implementation we landed on.
  • Where in standalone NodeGroup config should labels live? (If we copy the same struct, we will get spec.labels, while it might make more sense to have them as metadata.labels).
    • We will address the definition of the actual nodegroup config file later.
  • How will ClusterConfig and NodeGroup compare when inspected in a running cluster? Namely, are all nodegroups supposed to be inline in ClusterConfig as well as represented as distinct object? (This is currently hard to say, but in the future it will become important that there is coherent representation).
    • For the initial version, all nodeGroup definitions will live in the ClusterConfig