Toggle auditd #325

njuettner · 2024-09-06T06:07:27Z

I talked with Atlas and currently it's only needed for one customer. Enabling auditd leads to a lot of performance degradation (PTAL at the issue for further information) so we agreed to only enable it when needed.

By default we disable auditd:

ip-10-0-99-6 ~ # systemctl status auditd
○ auditd.service - Security Auditing Service
     Loaded: loaded (/usr/lib/systemd/system/auditd.service; disabled; preset: disabled)
     Active: inactive (dead)

Customer can enable it using global.components.audit.enabled=true

● auditd.service - Security Auditing Service
     Loaded: loaded (/usr/lib/systemd/system/auditd.service; enabled; preset: disabled)
     Active: active (running) since Fri 2024-09-06 09:34:31 UTC; 2min 58s ago

Should this change be mentioned in the release notes?

CHANGELOG.md has been updated (if it exists)

taylorbot · 2024-09-06T06:08:49Z

Hey @njuettner, a test pull request has been created for you in the cluster-aws repo! Go to pull request giantswarm/cluster-aws#816 in order to test your cluster chart changes on AWS.

taylorbot · 2024-09-06T07:34:13Z

Hey @njuettner, a test pull request has been created for you in the cluster-aws repo! Go to pull request giantswarm/cluster-aws#818 in order to test your cluster chart changes on AWS.

github-actions · 2024-09-06T09:52:13Z

There were differences in the rendered Helm template, please check! ⚠️

Output

=== Differences when rendered with values file helm/cluster/ci/test-required-values.yaml ===

(file level)
  - two documents removed:
    ---
    # Source: cluster/templates/clusterapi/workers/kubeadmconfig.yaml
    apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
    kind: KubeadmConfig
    metadata:
      name: awesome-def00-8f44c
      namespace: org-giantswarm
      annotations:
        machine-pool.giantswarm.io/name: awesome-def00
        important-cluster-value: 1000
        robots-need-this-in-the-cluster: eW91IGNhbm5vdCByZWFkIHRoaXMsIGJ1dCByb2JvdHMgY2FuCg==
        for-robots-in-nodepool: cm9ib3RzIGFyZSBvcGVyYXRpbmcgb24gdGhpcyBub2RlIHBvb2wK
      labels:
        # deprecated: "app: cluster-aws" label is deprecated and it will be removed after upgrading
    # to Kubernetes 1.25. We still need it here because existing ClusterResourceSet selectors
    # need this label on the Cluster resource.
    app: cluster-aws
        app.kubernetes.io/name: cluster
        app.kubernetes.io/version: 1.2.2
        app.kubernetes.io/part-of: cluster-aws
        app.kubernetes.io/instance: release-name
        app.kubernetes.io/managed-by: Helm
        helm.sh/chart: cluster-1.2.2
        application.giantswarm.io/team: turtles
        giantswarm.io/cluster: awesome
        giantswarm.io/organization: giantswarm
        giantswarm.io/service-priority: highest
        cluster.x-k8s.io/cluster-name: awesome
        cluster.x-k8s.io/watch-filter: capi
        release.giantswarm.io/version: 27.0.0-alpha.1
        another-cluster-label: label-2
        some-cluster-label: label-1
        giantswarm.io/machine-pool: awesome-def00
        nodepool-workload-type: ai
    spec:
      format: ignition
      ignition:
        containerLinuxConfig:
          additionalConfig: |
            systemd:
              units:      
              - name: os-hardening.service
                enabled: true
                contents: |
                  [Unit]
                  Description=Apply os hardening
                  [Service]
                  Type=oneshot
                  ExecStartPre=-/bin/bash -c "gpasswd -d core rkt; gpasswd -d core docker; gpasswd -d core wheel"
                  ExecStartPre=/bin/bash -c "until [ -f '/etc/sysctl.d/hardening.conf' ]; do echo Waiting for sysctl file; sleep 1s;done;"
                  ExecStart=/usr/sbin/sysctl -p /etc/sysctl.d/hardening.conf
                  [Install]
                  WantedBy=multi-user.target
              - name: update-engine.service
                enabled: false
                mask: true
              - name: locksmithd.service
                enabled: false
                mask: true
              - name: sshkeys.service
                enabled: false
                mask: true
              - name: teleport.service
                enabled: true
                contents: |
                  [Unit]
                  Description=Teleport Service
                  After=network.target
                  [Service]
                  Type=simple
                  Restart=on-failure
                  ExecStart=/opt/bin/teleport start --roles=node --config=/etc/teleport.yaml --pid-file=/run/teleport.pid
                  ExecReload=/bin/kill -HUP $MAINPID
                  PIDFile=/run/teleport.pid
                  LimitNOFILE=524288
                  [Install]
                  WantedBy=multi-user.target
              - name: kubeadm.service
                dropins:
                - name: 10-flatcar.conf
                  contents: |
                    [Unit]
                    # kubeadm must run after coreos-metadata populated /run/metadata directory.
                    Requires=coreos-metadata.service
                    After=coreos-metadata.service
                    # kubeadm must run after containerd - see https://github.com/kubernetes-sigs/image-builder/issues/939.
                    After=containerd.service
                    # kubeadm requires having an IP
                    After=network-online.target
                    Wants=network-online.target
                    [Service]
                    # Ensure kubeadm service has access to kubeadm binary in /opt/bin on Flatcar.
                    Environment=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/opt/bin
                    # To make metadata environment variables available for pre-kubeadm commands.
                    EnvironmentFile=/run/metadata/*
              - name: containerd.service
                enabled: true
                contents: |
                dropins:
                - name: 10-change-cgroup.conf
                  contents: |
                    [Service]
                    CPUAccounting=true
                    MemoryAccounting=true
                    Slice=kubereserved.slice
              - name: audit-rules.service
                enabled: true
                dropins:
                - name: 10-wait-for-containerd.conf
                  contents: |
                    [Service]
                    ExecStartPre=/bin/bash -c "while [ ! -f /etc/audit/rules.d/containerd.rules ]; do echo 'Waiting for /etc/audit/rules.d/containerd.rules to be written' && sleep 1; done"
                    Restart=on-failure      
              - name: var-lib-kubelet.mount
                enabled: true
                mask: false
                contents: |
                  [Unit]
                  Description=kubelet volume
                  DefaultDependencies=no
                  [Mount]
                  What=/dev/disk/by-label/kubelet
                  Where=/var/lib/kubelet
                  Type=xfs
                  [Install]
                  WantedBy=local-fs-pre.target
              - name: var-lib-containerd.mount
                enabled: true
                mask: false
                contents: |
                  [Unit]
                  Description=containerd volume
                  DefaultDependencies=no
                  [Mount]
                  What=/dev/disk/by-label/containerd
                  Where=/var/lib/containerd
                  Type=xfs
                  [Install]
                  WantedBy=local-fs-pre.target
              - name: example2.service
                enabled: false
                mask: false
                dropins:
                - name: hello1.conf
                  contents: |
                    # Multi-line
                    # contents goes here
                - name: hello2.conf
                  contents: |
                    # Multi-line
                    # contents goes here
              - name: var-lib-workload.mount
                enabled: true
                mask: false
                contents: |
                  [Unit]
                  Description=workload volume
                  DefaultDependencies=no
                  [Mount]
                  What=/dev/disk/by-label/workload
                  Where=/var/lib/workload
                  Type=xfs
                  [Install]
                  WantedBy=local-fs-pre.target
              - name: example2-workers.service
                enabled: false
                mask: false
                dropins:
                - name: hello1-workers.conf
                  contents: |
                    # Multi-line
                    # contents goes here
                - name: hello2-workers.conf
                  contents: |
                    # Multi-line
                    # contents goes here
            storage:
              filesystems:      
              directories:      
              - path: /var/lib/kubelet
                mode: 0750      
              - path: /var/lib/kubelet/temporary/stuff
                overwrite: true
                filesystem: kubelet
                mode: 750
                user:
                  id: 12345
                  name: giantswarm
                group:
                  id: 23456
                  name: giantswarm
              - path: /var/lib/kubelet/temporary/stuff/workers
                overwrite: true
                filesystem: kubelet
                mode: 750
                user:
                  id: 12345
                  name: giantswarm
                group:
                  id: 23456
                  name: giantswarm
            
      joinConfiguration:
        nodeRegistration:
          name: ${COREOS_EC2_HOSTNAME}
          kubeletExtraArgs:
            cgroup-driver: cgroupfs
            cloud-provider: external
            healthz-bind-address: 0.0.0.0
            node-ip: ${COREOS_EC2_IPV4_LOCAL}
            node-labels: "ip=${COREOS_EC2_IPV4_LOCAL},role=worker,giantswarm.io/machine-pool=awesome-def00,workload-type=ai"
            v: 2
          taints:
          - key: supernodepool
            value: hello
            effect: NoSchedule
        patches:
          directory: /etc/kubernetes/patches
      preKubeadmCommands:
      - "envsubst < /etc/kubeadm.yml > /etc/kubeadm.yml.tmp"
      - "mv /etc/kubeadm.yml.tmp /etc/kubeadm.yml"
      - "systemctl restart containerd"
      - "export HTTP_PROXY=http://proxy.giantswarm.io"
      - "export HTTPS_PROXY=https://proxy.giantswarm.io"
      - "export NO_PROXY="127.0.0.1,localhost,svc,local,awesome.example.gigantic.io,172.31.0.0/16,100.64.0.0/12,elb.amazonaws.com,169.254.169.254,some.noproxy.awesome.example.gigantic.io,another.noproxy.address.giantswarm.io,proxy1.example.com,proxy2.example.com""
      - "export http_proxy=http://proxy.giantswarm.io"
      - "export https_proxy=https://proxy.giantswarm.io"
      - "export no_proxy="127.0.0.1,localhost,svc,local,awesome.example.gigantic.io,172.31.0.0/16,100.64.0.0/12,elb.amazonaws.com,169.254.169.254,some.noproxy.awesome.example.gigantic.io,another.noproxy.address.giantswarm.io,proxy1.example.com,proxy2.example.com""
      - "echo "aws nodes command before kubeadm""
      - "echo "custom nodes command before kubeadm""
      - "echo "aws workers command before kubeadm""
      - "echo "custom workers command before kubeadm""
      postKubeadmCommands:
      - "echo "aws nodes command after kubeadm""
      - "echo "custom nodes command after kubeadm""
      - "echo "aws workers command after kubeadm""
      - "echo "custom workers command after kubeadm""
      files:
      - path: /etc/sysctl.d/hardening.conf
        permissions: 0644
        encoding: base64
        content: ZnMuaW5vdGlmeS5tYXhfdXNlcl93YXRjaGVzID0gMTYzODQKZnMuaW5vdGlmeS5tYXhfdXNlcl9pbnN0YW5jZXMgPSA4MTkyCmtlcm5lbC5rcHRyX3Jlc3RyaWN0ID0gMgprZXJuZWwuc3lzcnEgPSAwCm5ldC5pcHY0LmNvbmYuYWxsLmxvZ19tYXJ0aWFucyA9IDEKbmV0LmlwdjQuY29uZi5hbGwuc2VuZF9yZWRpcmVjdHMgPSAwCm5ldC5pcHY0LmNvbmYuZGVmYXVsdC5hY2NlcHRfcmVkaXJlY3RzID0gMApuZXQuaXB2NC5jb25mLmRlZmF1bHQubG9nX21hcnRpYW5zID0gMQpuZXQuaXB2NC50Y3BfdGltZXN0YW1wcyA9IDAKbmV0LmlwdjYuY29uZi5hbGwuYWNjZXB0X3JlZGlyZWN0cyA9IDAKbmV0LmlwdjYuY29uZi5kZWZhdWx0LmFjY2VwdF9yZWRpcmVjdHMgPSAwCiMgSW5jcmVhc2VkIG1tYXBmcyBiZWNhdXNlIHNvbWUgYXBwbGljYXRpb25zLCBsaWtlIEVTLCBuZWVkIGhpZ2hlciBsaW1pdCB0byBzdG9yZSBkYXRhIHByb3Blcmx5CnZtLm1heF9tYXBfY291bnQgPSAyNjIxNDQKIyBSZXNlcnZlZCB0byBhdm9pZCBjb25mbGljdHMgd2l0aCBrdWJlLWFwaXNlcnZlciwgd2hpY2ggYWxsb2NhdGVzIHdpdGhpbiB0aGlzIHJhbmdlCm5ldC5pcHY0LmlwX2xvY2FsX3Jlc2VydmVkX3BvcnRzPTMwMDAwLTMyNzY3Cm5ldC5pcHY0LmNvbmYuYWxsLnJwX2ZpbHRlciA9IDEKbmV0LmlwdjQuY29uZi5hbGwuYXJwX2lnbm9yZSA9IDEKbmV0LmlwdjQuY29uZi5hbGwuYXJwX2Fubm91bmNlID0gMgoKIyBUaGVzZSBhcmUgcmVxdWlyZWQgZm9yIHRoZSBrdWJlbGV0ICctLXByb3RlY3Qta2VybmVsLWRlZmF1bHRzJyBmbGFnCiMgU2VlIGh0dHBzOi8vZ2l0aHViLmNvbS9naWFudHN3YXJtL2dpYW50c3dhcm0vaXNzdWVzLzEzNTg3CnZtLm92ZXJjb21taXRfbWVtb3J5PTEKa2VybmVsLnBhbmljPTEwCmtlcm5lbC5wYW5pY19vbl9vb3BzPTEK
      - path: /etc/selinux/config
        permissions: 0644
        encoding: base64
        content: IyBUaGlzIGZpbGUgY29udHJvbHMgdGhlIHN0YXRlIG9mIFNFTGludXggb24gdGhlIHN5c3RlbSBvbiBib290LgoKIyBTRUxJTlVYIGNhbiB0YWtlIG9uZSBvZiB0aGVzZSB0aHJlZSB2YWx1ZXM6CiMgICAgICAgZW5mb3JjaW5nIC0gU0VMaW51eCBzZWN1cml0eSBwb2xpY3kgaXMgZW5mb3JjZWQuCiMgICAgICAgcGVybWlzc2l2ZSAtIFNFTGludXggcHJpbnRzIHdhcm5pbmdzIGluc3RlYWQgb2YgZW5mb3JjaW5nLgojICAgICAgIGRpc2FibGVkIC0gTm8gU0VMaW51eCBwb2xpY3kgaXMgbG9hZGVkLgpTRUxJTlVYPXBlcm1pc3NpdmUKCiMgU0VMSU5VWFRZUEUgY2FuIHRha2Ugb25lIG9mIHRoZXNlIGZvdXIgdmFsdWVzOgojICAgICAgIHRhcmdldGVkIC0gT25seSB0YXJnZXRlZCBuZXR3b3JrIGRhZW1vbnMgYXJlIHByb3RlY3RlZC4KIyAgICAgICBzdHJpY3QgICAtIEZ1bGwgU0VMaW51eCBwcm90ZWN0aW9uLgojICAgICAgIG1scyAgICAgIC0gRnVsbCBTRUxpbnV4IHByb3RlY3Rpb24gd2l0aCBNdWx0aS1MZXZlbCBTZWN1cml0eQojICAgICAgIG1jcyAgICAgIC0gRnVsbCBTRUxpbnV4IHByb3RlY3Rpb24gd2l0aCBNdWx0aS1DYXRlZ29yeSBTZWN1cml0eQojICAgICAgICAgICAgICAgICAgKG1scywgYnV0IG9ubHkgb25lIHNlbnNpdGl2aXR5IGxldmVsKQpTRUxJTlVYVFlQRT1tY3MK
      - path: /etc/systemd/timesyncd.conf
        permissions: 0644
        encoding: base64
        content: W1RpbWVdCk5UUD0xNjkuMjU0LjE2OS4xMjMK
      - path: /etc/flatcar-cgroupv1
        filesystem: root
        permissions: 0444
      - path: /etc/containerd/config.toml
        permissions: 0644
        contentFrom:
          secret:
            name: awesome-containerd-dbb9d89e
            key: config.toml
      - path: /etc/kubernetes/patches/kubeletconfiguration.yaml
        permissions: 0644
        encoding: base64
        content: YXBpVmVyc2lvbjoga3ViZWxldC5jb25maWcuazhzLmlvL3YxYmV0YTEKa2luZDogS3ViZWxldENvbmZpZ3VyYXRpb24Kc2h1dGRvd25HcmFjZVBlcmlvZDogMzAwcwpzaHV0ZG93bkdyYWNlUGVyaW9kQ3JpdGljYWxQb2RzOiA2MHMKa2VybmVsTWVtY2dOb3RpZmljYXRpb246IHRydWUKZXZpY3Rpb25Tb2Z0OgogIG1lbW9yeS5hdmFpbGFibGU6ICI1MDBNaSIKZXZpY3Rpb25IYXJkOgogIG1lbW9yeS5hdmFpbGFibGU6ICIyMDBNaSIKICBpbWFnZWZzLmF2YWlsYWJsZTogIjE1JSIKZXZpY3Rpb25Tb2Z0R3JhY2VQZXJpb2Q6CiAgbWVtb3J5LmF2YWlsYWJsZTogIjVzIgpldmljdGlvbk1heFBvZEdyYWNlUGVyaW9kOiA2MAprdWJlUmVzZXJ2ZWQ6CiAgY3B1OiAzNTBtCiAgbWVtb3J5OiAzODRNaQogIGVwaGVtZXJhbC1zdG9yYWdlOiAxMDI0TWkKa3ViZVJlc2VydmVkQ2dyb3VwOiAva3ViZXJlc2VydmVkLnNsaWNlCnByb3RlY3RLZXJuZWxEZWZhdWx0czogdHJ1ZQpzeXN0ZW1SZXNlcnZlZDoKICBjcHU6IDI1MG0KICBtZW1vcnk6IDEyODBNaQpzeXN0ZW1SZXNlcnZlZENncm91cDogL3N5c3RlbS5zbGljZQp0bHNDaXBoZXJTdWl0ZXM6Ci0gVExTX0FFU18xMjhfR0NNX1NIQTI1NgotIFRMU19BRVNfMjU2X0dDTV9TSEEzODQKLSBUTFNfQ0hBQ0hBMjBfUE9MWTEzMDVfU0hBMjU2Ci0gVExTX0VDREhFX0VDRFNBX1dJVEhfQUVTXzEyOF9DQkNfU0hBCi0gVExTX0VDREhFX0VDRFNBX1dJVEhfQUVTXzEyOF9HQ01fU0hBMjU2Ci0gVExTX0VDREhFX0VDRFNBX1dJVEhfQUVTXzI1Nl9DQkNfU0hBCi0gVExTX0VDREhFX0VDRFNBX1dJVEhfQUVTXzI1Nl9HQ01fU0hBMzg0Ci0gVExTX0VDREhFX0VDRFNBX1dJVEhfQ0hBQ0hBMjBfUE9MWTEzMDUKLSBUTFNfRUNESEVfRUNEU0FfV0lUSF9DSEFDSEEyMF9QT0xZMTMwNV9TSEEyNTYKLSBUTFNfRUNESEVfUlNBX1dJVEhfQUVTXzEyOF9DQkNfU0hBCi0gVExTX0VDREhFX1JTQV9XSVRIX0FFU18xMjhfR0NNX1NIQTI1NgotIFRMU19FQ0RIRV9SU0FfV0lUSF9BRVNfMjU2X0NCQ19TSEEKLSBUTFNfRUNESEVfUlNBX1dJVEhfQUVTXzI1Nl9HQ01fU0hBMzg0Ci0gVExTX0VDREhFX1JTQV9XSVRIX0NIQUNIQTIwX1BPTFkxMzA1Ci0gVExTX0VDREhFX1JTQV9XSVRIX0NIQUNIQTIwX1BPTFkxMzA1X1NIQTI1NgotIFRMU19SU0FfV0lUSF9BRVNfMTI4X0NCQ19TSEEKLSBUTFNfUlNBX1dJVEhfQUVTXzEyOF9HQ01fU0hBMjU2Ci0gVExTX1JTQV9XSVRIX0FFU18yNTZfQ0JDX1NIQQotIFRMU19SU0FfV0lUSF9BRVNfMjU2X0dDTV9TSEEzODQKc2VyaWFsaXplSW1hZ2VQdWxsczogZmFsc2UKc3RyZWFtaW5nQ29ubmVjdGlvbklkbGVUaW1lb3V0OiAxaAphbGxvd2VkVW5zYWZlU3lzY3RsczoKLSAibmV0LioiCmNvbnRhaW5lckxvZ01heFNpemU6IDMwTWkKY29udGFpbmVyTG9nTWF4RmlsZXM6IDIK
      - path: /etc/systemd/logind.conf.d/zzz-kubelet-graceful-shutdown.conf
        permissions: 0700
        encoding: base64
        content: W0xvZ2luXQojIGRlbGF5CkluaGliaXREZWxheU1heFNlYz0zMDAK
      - path: /etc/systemd/system/containerd.service.d/http-proxy.conf
        permissions: 0644
        encoding: base64
        content: W1NlcnZpY2VdCkVudmlyb25tZW50PSJIVFRQX1BST1hZPWh0dHA6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0iSFRUUFNfUFJPWFk9aHR0cHM6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0iTk9fUFJPWFk9MTI3LjAuMC4xLGxvY2FsaG9zdCxzdmMsbG9jYWwsYXdlc29tZS5leGFtcGxlLmdpZ2FudGljLmlvLDE3Mi4zMS4wLjAvMTYsMTAwLjY0LjAuMC8xMixlbGIuYW1hem9uYXdzLmNvbSwxNjkuMjU0LjE2OS4yNTQsc29tZS5ub3Byb3h5LmF3ZXNvbWUuZXhhbXBsZS5naWdhbnRpYy5pbyxhbm90aGVyLm5vcHJveHkuYWRkcmVzcy5naWFudHN3YXJtLmlvLHByb3h5MS5leGFtcGxlLmNvbSxwcm94eTIuZXhhbXBsZS5jb20iCkVudmlyb25tZW50PSJodHRwX3Byb3h5PWh0dHA6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0iaHR0cHNfcHJveHk9aHR0cHM6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0ibm9fcHJveHk9MTI3LjAuMC4xLGxvY2FsaG9zdCxzdmMsbG9jYWwsYXdlc29tZS5leGFtcGxlLmdpZ2FudGljLmlvLDE3Mi4zMS4wLjAvMTYsMTAwLjY0LjAuMC8xMixlbGIuYW1hem9uYXdzLmNvbSwxNjkuMjU0LjE2OS4yNTQsc29tZS5ub3Byb3h5LmF3ZXNvbWUuZXhhbXBsZS5naWdhbnRpYy5pbyxhbm90aGVyLm5vcHJveHkuYWRkcmVzcy5naWFudHN3YXJtLmlvLHByb3h5MS5leGFtcGxlLmNvbSxwcm94eTIuZXhhbXBsZS5jb20iCg==
      - path: /etc/systemd/system/kubelet.service.d/http-proxy.conf
        permissions: 0644
        encoding: base64
        content: W1NlcnZpY2VdCkVudmlyb25tZW50PSJIVFRQX1BST1hZPWh0dHA6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0iSFRUUFNfUFJPWFk9aHR0cHM6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0iTk9fUFJPWFk9MTI3LjAuMC4xLGxvY2FsaG9zdCxzdmMsbG9jYWwsYXdlc29tZS5leGFtcGxlLmdpZ2FudGljLmlvLDE3Mi4zMS4wLjAvMTYsMTAwLjY0LjAuMC8xMixlbGIuYW1hem9uYXdzLmNvbSwxNjkuMjU0LjE2OS4yNTQsc29tZS5ub3Byb3h5LmF3ZXNvbWUuZXhhbXBsZS5naWdhbnRpYy5pbyxhbm90aGVyLm5vcHJveHkuYWRkcmVzcy5naWFudHN3YXJtLmlvLHByb3h5MS5leGFtcGxlLmNvbSxwcm94eTIuZXhhbXBsZS5jb20iCkVudmlyb25tZW50PSJodHRwX3Byb3h5PWh0dHA6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0iaHR0cHNfcHJveHk9aHR0cHM6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0ibm9fcHJveHk9MTI3LjAuMC4xLGxvY2FsaG9zdCxzdmMsbG9jYWwsYXdlc29tZS5leGFtcGxlLmdpZ2FudGljLmlvLDE3Mi4zMS4wLjAvMTYsMTAwLjY0LjAuMC8xMixlbGIuYW1hem9uYXdzLmNvbSwxNjkuMjU0LjE2OS4yNTQsc29tZS5ub3Byb3h5LmF3ZXNvbWUuZXhhbXBsZS5naWdhbnRpYy5pbyxhbm90aGVyLm5vcHJveHkuYWRkcmVzcy5naWFudHN3YXJtLmlvLHByb3h5MS5leGFtcGxlLmNvbSxwcm94eTIuZXhhbXBsZS5jb20iCg==
      - path: /etc/systemd/system/teleport.service.d/http-proxy.conf
        permissions: 0644
        encoding: base64
        content: W1NlcnZpY2VdCkVudmlyb25tZW50PSJIVFRQX1BST1hZPWh0dHA6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0iSFRUUFNfUFJPWFk9aHR0cHM6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0iTk9fUFJPWFk9MTI3LjAuMC4xLGxvY2FsaG9zdCxzdmMsbG9jYWwsYXdlc29tZS5leGFtcGxlLmdpZ2FudGljLmlvLDE3Mi4zMS4wLjAvMTYsMTAwLjY0LjAuMC8xMixlbGIuYW1hem9uYXdzLmNvbSwxNjkuMjU0LjE2OS4yNTQsc29tZS5ub3Byb3h5LmF3ZXNvbWUuZXhhbXBsZS5naWdhbnRpYy5pbyxhbm90aGVyLm5vcHJveHkuYWRkcmVzcy5naWFudHN3YXJtLmlvLHByb3h5MS5leGFtcGxlLmNvbSxwcm94eTIuZXhhbXBsZS5jb20iCkVudmlyb25tZW50PSJodHRwX3Byb3h5PWh0dHA6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0iaHR0cHNfcHJveHk9aHR0cHM6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0ibm9fcHJveHk9MTI3LjAuMC4xLGxvY2FsaG9zdCxzdmMsbG9jYWwsYXdlc29tZS5leGFtcGxlLmdpZ2FudGljLmlvLDE3Mi4zMS4wLjAvMTYsMTAwLjY0LjAuMC8xMixlbGIuYW1hem9uYXdzLmNvbSwxNjkuMjU0LjE2OS4yNTQsc29tZS5ub3Byb3h5LmF3ZXNvbWUuZXhhbXBsZS5naWdhbnRpYy5pbyxhbm90aGVyLm5vcHJveHkuYWRkcmVzcy5naWFudHN3YXJtLmlvLHByb3h5MS5leGFtcGxlLmNvbSxwcm94eTIuZXhhbXBsZS5jb20iCg==
      - path: /etc/teleport-join-token
        permissions: 0644
        contentFrom:
          secret:
            name: awesome-teleport-join-token
            key: joinToken
      - path: /opt/teleport-node-role.sh
        permissions: 0755
        encoding: base64
        content: IyEvYmluL2Jhc2gKCmlmIHN5c3RlbWN0bCBpcy1hY3RpdmUgLXEga3ViZWxldC5zZXJ2aWNlOyB0aGVuCiAgICBpZiBbIC1lICIvZXRjL2t1YmVybmV0ZXMvbWFuaWZlc3RzL2t1YmUtYXBpc2VydmVyLnlhbWwiIF07IHRoZW4KICAgICAgICBlY2hvICJjb250cm9sLXBsYW5lIgogICAgZWxzZQogICAgICAgIGVjaG8gIndvcmtlciIKICAgIGZpCmVsc2UKICAgIGVjaG8gIiIKZmkK
      - path: /etc/teleport.yaml
        permissions: 0644
        encoding: base64
        content: dmVyc2lvbjogdjMKdGVsZXBvcnQ6CiAgZGF0YV9kaXI6IC92YXIvbGliL3RlbGVwb3J0CiAgam9pbl9wYXJhbXM6CiAgICB0b2tlbl9uYW1lOiAvZXRjL3RlbGVwb3J0LWpvaW4tdG9rZW4KICAgIG1ldGhvZDogdG9rZW4KICBwcm94eV9zZXJ2ZXI6IHRlbGVwb3J0LmdpYW50c3dhcm0uaW86NDQzCiAgbG9nOgogICAgb3V0cHV0OiBzdGRlcnIKYXV0aF9zZXJ2aWNlOgogIGVuYWJsZWQ6ICJubyIKc3NoX3NlcnZpY2U6CiAgZW5hYmxlZDogInllcyIKICBjb21tYW5kczoKICAtIG5hbWU6IG5vZGUKICAgIGNvbW1hbmQ6IFtob3N0bmFtZV0KICAgIHBlcmlvZDogMjRoMG0wcwogIC0gbmFtZTogYXJjaAogICAgY29tbWFuZDogW3VuYW1lLCAtbV0KICAgIHBlcmlvZDogMjRoMG0wcwogIC0gbmFtZTogcm9sZQogICAgY29tbWFuZDogWy9vcHQvdGVsZXBvcnQtbm9kZS1yb2xlLnNoXQogICAgcGVyaW9kOiAxbTBzCiAgbGFiZWxzOgogICAgaW5zOiBnaWFudG1jCiAgICBtYzogZ2lhbnRtYwogICAgY2x1c3RlcjogYXdlc29tZQogICAgYmFzZURvbWFpbjogZXhhbXBsZS5naWdhbnRpYy5pbwpwcm94eV9zZXJ2aWNlOgogIGVuYWJsZWQ6ICJubyIK
      - path: /etc/audit/rules.d/99-default.rules
        permissions: 0640
        encoding: base64
        content: IyBPdmVycmlkZGVuIGJ5IEdpYW50IFN3YXJtLgotYSBleGl0LGFsd2F5cyAtRiBhcmNoPWI2NCAtUyBleGVjdmUgLWsgYXVkaXRpbmcKLWEgZXhpdCxhbHdheXMgLUYgYXJjaD1iMzIgLVMgZXhlY3ZlIC1rIGF1ZGl0aW5nCg==
      - contentFrom:
          secret:
            name: cluster-super-secret
            key: node-stuff
        path: /etc/aws/node/file.yaml
        permissions: 0644
      - contentFrom:
          secret:
            name: cluster-super-secret
            key: node-stuff
        path: /etc/custom/node/file.yaml
        permissions: 0644
      - contentFrom:
          secret:
            name: cluster-super-secret-worker
            key: node-stuff
        path: /etc/aws/worker/node/file.yaml
        permissions: 0644
      - contentFrom:
          secret:
            name: cluster-super-secret-worker
            key: node-stuff
        path: /etc/custom/worker/node/file.yaml
        permissions: 0644
    # Source: cluster/templates/clusterapi/workers/kubeadmconfig.yaml
    apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
    kind: KubeadmConfig
    metadata:
      annotations:
        machine-pool.giantswarm.io/name: awesome-verybignodepool-1234
        important-cluster-value: 1000
        robots-need-this-in-the-cluster: eW91IGNhbm5vdCByZWFkIHRoaXMsIGJ1dCByb2JvdHMgY2FuCg==
      labels:
        # deprecated: "app: cluster-aws" label is deprecated and it will be removed after upgrading
    # to Kubernetes 1.25. We still need it here because existing ClusterResourceSet selectors
    # need this label on the Cluster resource.
    app: cluster-aws
        app.kubernetes.io/name: cluster
        app.kubernetes.io/version: 1.2.2
        app.kubernetes.io/part-of: cluster-aws
        app.kubernetes.io/instance: release-name
        app.kubernetes.io/managed-by: Helm
        helm.sh/chart: cluster-1.2.2
        application.giantswarm.io/team: turtles
        giantswarm.io/cluster: awesome
        giantswarm.io/organization: giantswarm
        giantswarm.io/service-priority: highest
        cluster.x-k8s.io/cluster-name: awesome
        cluster.x-k8s.io/watch-filter: capi
        release.giantswarm.io/version: 27.0.0-alpha.1
        another-cluster-label: label-2
        some-cluster-label: label-1
        giantswarm.io/machine-pool: awesome-verybignodepool-1234
      name: awesome-verybignodepool-1234-1b3cf
      namespace: org-giantswarm
    spec:
      format: ignition
      ignition:
        containerLinuxConfig:
          additionalConfig: |
            systemd:
              units:      
              - name: os-hardening.service
                enabled: true
                contents: |
                  [Unit]
                  Description=Apply os hardening
                  [Service]
                  Type=oneshot
                  ExecStartPre=-/bin/bash -c "gpasswd -d core rkt; gpasswd -d core docker; gpasswd -d core wheel"
                  ExecStartPre=/bin/bash -c "until [ -f '/etc/sysctl.d/hardening.conf' ]; do echo Waiting for sysctl file; sleep 1s;done;"
                  ExecStart=/usr/sbin/sysctl -p /etc/sysctl.d/hardening.conf
                  [Install]
                  WantedBy=multi-user.target
              - name: update-engine.service
                enabled: false
                mask: true
              - name: locksmithd.service
                enabled: false
                mask: true
              - name: sshkeys.service
                enabled: false
                mask: true
              - name: teleport.service
                enabled: true
                contents: |
                  [Unit]
                  Description=Teleport Service
                  After=network.target
                  [Service]
                  Type=simple
                  Restart=on-failure
                  ExecStart=/opt/bin/teleport start --roles=node --config=/etc/teleport.yaml --pid-file=/run/teleport.pid
                  ExecReload=/bin/kill -HUP $MAINPID
                  PIDFile=/run/teleport.pid
                  LimitNOFILE=524288
                  [Install]
                  WantedBy=multi-user.target
              - name: kubeadm.service
                dropins:
                - name: 10-flatcar.conf
                  contents: |
                    [Unit]
                    # kubeadm must run after coreos-metadata populated /run/metadata directory.
                    Requires=coreos-metadata.service
                    After=coreos-metadata.service
                    # kubeadm must run after containerd - see https://github.com/kubernetes-sigs/image-builder/issues/939.
                    After=containerd.service
                    # kubeadm requires having an IP
                    After=network-online.target
                    Wants=network-online.target
                    [Service]
                    # Ensure kubeadm service has access to kubeadm binary in /opt/bin on Flatcar.
                    Environment=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/opt/bin
                    # To make metadata environment variables available for pre-kubeadm commands.
                    EnvironmentFile=/run/metadata/*
              - name: containerd.service
                enabled: true
                contents: |
                dropins:
                - name: 10-change-cgroup.conf
                  contents: |
                    [Service]
                    CPUAccounting=true
                    MemoryAccounting=true
                    Slice=kubereserved.slice
              - name: audit-rules.service
                enabled: true
                dropins:
                - name: 10-wait-for-containerd.conf
                  contents: |
                    [Service]
                    ExecStartPre=/bin/bash -c "while [ ! -f /etc/audit/rules.d/containerd.rules ]; do echo 'Waiting for /etc/audit/rules.d/containerd.rules to be written' && sleep 1; done"
                    Restart=on-failure      
              - name: var-lib-kubelet.mount
                enabled: true
                mask: false
                contents: |
                  [Unit]
                  Description=kubelet volume
                  DefaultDependencies=no
                  [Mount]
                  What=/dev/disk/by-label/kubelet
                  Where=/var/lib/kubelet
                  Type=xfs
                  [Install]
                  WantedBy=local-fs-pre.target
              - name: var-lib-containerd.mount
                enabled: true
                mask: false
                contents: |
                  [Unit]
                  Description=containerd volume
                  DefaultDependencies=no
                  [Mount]
                  What=/dev/disk/by-label/containerd
                  Where=/var/lib/containerd
                  Type=xfs
                  [Install]
                  WantedBy=local-fs-pre.target
              - name: example2.service
                enabled: false
                mask: false
                dropins:
                - name: hello1.conf
                  contents: |
                    # Multi-line
                    # contents goes here
                - name: hello2.conf
                  contents: |
                    # Multi-line
                    # contents goes here
              - name: var-lib-workload.mount
                enabled: true
                mask: false
                contents: |
                  [Unit]
                  Description=workload volume
                  DefaultDependencies=no
                  [Mount]
                  What=/dev/disk/by-label/workload
                  Where=/var/lib/workload
                  Type=xfs
                  [Install]
                  WantedBy=local-fs-pre.target
              - name: example2-workers.service
                enabled: false
                mask: false
                dropins:
                - name: hello1-workers.conf
                  contents: |
                    # Multi-line
                    # contents goes here
                - name: hello2-workers.conf
                  contents: |
                    # Multi-line
                    # contents goes here
            storage:
              filesystems:      
              directories:      
              - path: /var/lib/kubelet
                mode: 0750      
              - path: /var/lib/kubelet/temporary/stuff
                overwrite: true
                filesystem: kubelet
                mode: 750
                user:
                  id: 12345
                  name: giantswarm
                group:
                  id: 23456
                  name: giantswarm
              - path: /var/lib/kubelet/temporary/stuff/workers
                overwrite: true
                filesystem: kubelet
                mode: 750
                user:
                  id: 12345
                  name: giantswarm
                group:
                  id: 23456
                  name: giantswarm
            
      joinConfiguration:
        nodeRegistration:
          name: ${COREOS_EC2_HOSTNAME}
          kubeletExtraArgs:
            cgroup-driver: cgroupfs
            cloud-provider: external
            healthz-bind-address: 0.0.0.0
            node-ip: ${COREOS_EC2_IPV4_LOCAL}
            node-labels: "ip=${COREOS_EC2_IPV4_LOCAL},role=worker,giantswarm.io/machine-pool=awesome-verybignodepool-1234,workload-type=robots"
            v: 2
        patches:
          directory: /etc/kubernetes/patches
      preKubeadmCommands:
      - "envsubst < /etc/kubeadm.yml > /etc/kubeadm.yml.tmp"
      - "mv /etc/kubeadm.yml.tmp /etc/kubeadm.yml"
      - "systemctl restart containerd"
      - "export HTTP_PROXY=http://proxy.giantswarm.io"
      - "export HTTPS_PROXY=https://proxy.giantswarm.io"
      - "export NO_PROXY="127.0.0.1,localhost,svc,local,awesome.example.gigantic.io,172.31.0.0/16,100.64.0.0/12,elb.amazonaws.com,169.254.169.254,some.noproxy.awesome.example.gigantic.io,another.noproxy.address.giantswarm.io,proxy1.example.com,proxy2.example.com""
      - "export http_proxy=http://proxy.giantswarm.io"
      - "export https_proxy=https://proxy.giantswarm.io"
      - "export no_proxy="127.0.0.1,localhost,svc,local,awesome.example.gigantic.io,172.31.0.0/16,100.64.0.0/12,elb.amazonaws.com,169.254.169.254,some.noproxy.awesome.example.gigantic.io,another.noproxy.address.giantswarm.io,proxy1.example.com,proxy2.example.com""
      - "echo "aws nodes command before kubeadm""
      - "echo "custom nodes command before kubeadm""
      - "echo "aws workers command before kubeadm""
      - "echo "custom workers command before kubeadm""
      postKubeadmCommands:
      - "echo "aws nodes command after kubeadm""
      - "echo "custom nodes command after kubeadm""
      - "echo "aws workers command after kubeadm""
      - "echo "custom workers command after kubeadm""
      files:
      - path: /etc/sysctl.d/hardening.conf
        permissions: 0644
        encoding: base64
        content: ZnMuaW5vdGlmeS5tYXhfdXNlcl93YXRjaGVzID0gMTYzODQKZnMuaW5vdGlmeS5tYXhfdXNlcl9pbnN0YW5jZXMgPSA4MTkyCmtlcm5lbC5rcHRyX3Jlc3RyaWN0ID0gMgprZXJuZWwuc3lzcnEgPSAwCm5ldC5pcHY0LmNvbmYuYWxsLmxvZ19tYXJ0aWFucyA9IDEKbmV0LmlwdjQuY29uZi5hbGwuc2VuZF9yZWRpcmVjdHMgPSAwCm5ldC5pcHY0LmNvbmYuZGVmYXVsdC5hY2NlcHRfcmVkaXJlY3RzID0gMApuZXQuaXB2NC5jb25mLmRlZmF1bHQubG9nX21hcnRpYW5zID0gMQpuZXQuaXB2NC50Y3BfdGltZXN0YW1wcyA9IDAKbmV0LmlwdjYuY29uZi5hbGwuYWNjZXB0X3JlZGlyZWN0cyA9IDAKbmV0LmlwdjYuY29uZi5kZWZhdWx0LmFjY2VwdF9yZWRpcmVjdHMgPSAwCiMgSW5jcmVhc2VkIG1tYXBmcyBiZWNhdXNlIHNvbWUgYXBwbGljYXRpb25zLCBsaWtlIEVTLCBuZWVkIGhpZ2hlciBsaW1pdCB0byBzdG9yZSBkYXRhIHByb3Blcmx5CnZtLm1heF9tYXBfY291bnQgPSAyNjIxNDQKIyBSZXNlcnZlZCB0byBhdm9pZCBjb25mbGljdHMgd2l0aCBrdWJlLWFwaXNlcnZlciwgd2hpY2ggYWxsb2NhdGVzIHdpdGhpbiB0aGlzIHJhbmdlCm5ldC5pcHY0LmlwX2xvY2FsX3Jlc2VydmVkX3BvcnRzPTMwMDAwLTMyNzY3Cm5ldC5pcHY0LmNvbmYuYWxsLnJwX2ZpbHRlciA9IDEKbmV0LmlwdjQuY29uZi5hbGwuYXJwX2lnbm9yZSA9IDEKbmV0LmlwdjQuY29uZi5hbGwuYXJwX2Fubm91bmNlID0gMgoKIyBUaGVzZSBhcmUgcmVxdWlyZWQgZm9yIHRoZSBrdWJlbGV0ICctLXByb3RlY3Qta2VybmVsLWRlZmF1bHRzJyBmbGFnCiMgU2VlIGh0dHBzOi8vZ2l0aHViLmNvbS9naWFudHN3YXJtL2dpYW50c3dhcm0vaXNzdWVzLzEzNTg3CnZtLm92ZXJjb21taXRfbWVtb3J5PTEKa2VybmVsLnBhbmljPTEwCmtlcm5lbC5wYW5pY19vbl9vb3BzPTEK
      - path: /etc/selinux/config
        permissions: 0644
        encoding: base64
        content: IyBUaGlzIGZpbGUgY29udHJvbHMgdGhlIHN0YXRlIG9mIFNFTGludXggb24gdGhlIHN5c3RlbSBvbiBib290LgoKIyBTRUxJTlVYIGNhbiB0YWtlIG9uZSBvZiB0aGVzZSB0aHJlZSB2YWx1ZXM6CiMgICAgICAgZW5mb3JjaW5nIC0gU0VMaW51eCBzZWN1cml0eSBwb2xpY3kgaXMgZW5mb3JjZWQuCiMgICAgICAgcGVybWlzc2l2ZSAtIFNFTGludXggcHJpbnRzIHdhcm5pbmdzIGluc3RlYWQgb2YgZW5mb3JjaW5nLgojICAgICAgIGRpc2FibGVkIC0gTm8gU0VMaW51eCBwb2xpY3kgaXMgbG9hZGVkLgpTRUxJTlVYPXBlcm1pc3NpdmUKCiMgU0VMSU5VWFRZUEUgY2FuIHRha2Ugb25lIG9mIHRoZXNlIGZvdXIgdmFsdWVzOgojICAgICAgIHRhcmdldGVkIC0gT25seSB0YXJnZXRlZCBuZXR3b3JrIGRhZW1vbnMgYXJlIHByb3RlY3RlZC4KIyAgICAgICBzdHJpY3QgICAtIEZ1bGwgU0VMaW51eCBwcm90ZWN0aW9uLgojICAgICAgIG1scyAgICAgIC0gRnVsbCBTRUxpbnV4IHByb3RlY3Rpb24gd2l0aCBNdWx0aS1MZXZlbCBTZWN1cml0eQojICAgICAgIG1jcyAgICAgIC0gRnVsbCBTRUxpbnV4IHByb3RlY3Rpb24gd2l0aCBNdWx0aS1DYXRlZ29yeSBTZWN1cml0eQojICAgICAgICAgICAgICAgICAgKG1scywgYnV0IG9ubHkgb25lIHNlbnNpdGl2aXR5IGxldmVsKQpTRUxJTlVYVFlQRT1tY3MK
      - path: /etc/systemd/timesyncd.conf
        permissions: 0644
        encoding: base64
        content: W1RpbWVdCk5UUD0xNjkuMjU0LjE2OS4xMjMK
      - path: /etc/flatcar-cgroupv1
        filesystem: root
        permissions: 0444
      - path: /etc/containerd/config.toml
        permissions: 0644
        contentFrom:
          secret:
            name: awesome-containerd-dbb9d89e
            key: config.toml
      - path: /etc/kubernetes/patches/kubeletconfiguration.yaml
        permissions: 0644
        encoding: base64
        content: YXBpVmVyc2lvbjoga3ViZWxldC5jb25maWcuazhzLmlvL3YxYmV0YTEKa2luZDogS3ViZWxldENvbmZpZ3VyYXRpb24Kc2h1dGRvd25HcmFjZVBlcmlvZDogMzAwcwpzaHV0ZG93bkdyYWNlUGVyaW9kQ3JpdGljYWxQb2RzOiA2MHMKa2VybmVsTWVtY2dOb3RpZmljYXRpb246IHRydWUKZXZpY3Rpb25Tb2Z0OgogIG1lbW9yeS5hdmFpbGFibGU6ICI1MDBNaSIKZXZpY3Rpb25IYXJkOgogIG1lbW9yeS5hdmFpbGFibGU6ICIyMDBNaSIKICBpbWFnZWZzLmF2YWlsYWJsZTogIjE1JSIKZXZpY3Rpb25Tb2Z0R3JhY2VQZXJpb2Q6CiAgbWVtb3J5LmF2YWlsYWJsZTogIjVzIgpldmljdGlvbk1heFBvZEdyYWNlUGVyaW9kOiA2MAprdWJlUmVzZXJ2ZWQ6CiAgY3B1OiAzNTBtCiAgbWVtb3J5OiAzODRNaQogIGVwaGVtZXJhbC1zdG9yYWdlOiAxMDI0TWkKa3ViZVJlc2VydmVkQ2dyb3VwOiAva3ViZXJlc2VydmVkLnNsaWNlCnByb3RlY3RLZXJuZWxEZWZhdWx0czogdHJ1ZQpzeXN0ZW1SZXNlcnZlZDoKICBjcHU6IDI1MG0KICBtZW1vcnk6IDEyODBNaQpzeXN0ZW1SZXNlcnZlZENncm91cDogL3N5c3RlbS5zbGljZQp0bHNDaXBoZXJTdWl0ZXM6Ci0gVExTX0FFU18xMjhfR0NNX1NIQTI1NgotIFRMU19BRVNfMjU2X0dDTV9TSEEzODQKLSBUTFNfQ0hBQ0hBMjBfUE9MWTEzMDVfU0hBMjU2Ci0gVExTX0VDREhFX0VDRFNBX1dJVEhfQUVTXzEyOF9DQkNfU0hBCi0gVExTX0VDREhFX0VDRFNBX1dJVEhfQUVTXzEyOF9HQ01fU0hBMjU2Ci0gVExTX0VDREhFX0VDRFNBX1dJVEhfQUVTXzI1Nl9DQkNfU0hBCi0gVExTX0VDREhFX0VDRFNBX1dJVEhfQUVTXzI1Nl9HQ01fU0hBMzg0Ci0gVExTX0VDREhFX0VDRFNBX1dJVEhfQ0hBQ0hBMjBfUE9MWTEzMDUKLSBUTFNfRUNESEVfRUNEU0FfV0lUSF9DSEFDSEEyMF9QT0xZMTMwNV9TSEEyNTYKLSBUTFNfRUNESEVfUlNBX1dJVEhfQUVTXzEyOF9DQkNfU0hBCi0gVExTX0VDREhFX1JTQV9XSVRIX0FFU18xMjhfR0NNX1NIQTI1NgotIFRMU19FQ0RIRV9SU0FfV0lUSF9BRVNfMjU2X0NCQ19TSEEKLSBUTFNfRUNESEVfUlNBX1dJVEhfQUVTXzI1Nl9HQ01fU0hBMzg0Ci0gVExTX0VDREhFX1JTQV9XSVRIX0NIQUNIQTIwX1BPTFkxMzA1Ci0gVExTX0VDREhFX1JTQV9XSVRIX0NIQUNIQTIwX1BPTFkxMzA1X1NIQTI1NgotIFRMU19SU0FfV0lUSF9BRVNfMTI4X0NCQ19TSEEKLSBUTFNfUlNBX1dJVEhfQUVTXzEyOF9HQ01fU0hBMjU2Ci0gVExTX1JTQV9XSVRIX0FFU18yNTZfQ0JDX1NIQQotIFRMU19SU0FfV0lUSF9BRVNfMjU2X0dDTV9TSEEzODQKc2VyaWFsaXplSW1hZ2VQdWxsczogZmFsc2UKc3RyZWFtaW5nQ29ubmVjdGlvbklkbGVUaW1lb3V0OiAxaAphbGxvd2VkVW5zYWZlU3lzY3RsczoKLSAibmV0LioiCmNvbnRhaW5lckxvZ01heFNpemU6IDMwTWkKY29udGFpbmVyTG9nTWF4RmlsZXM6IDIK
      - path: /etc/systemd/logind.conf.d/zzz-kubelet-graceful-shutdown.conf
        permissions: 0700
        encoding: base64
        content: W0xvZ2luXQojIGRlbGF5CkluaGliaXREZWxheU1heFNlYz0zMDAK
      - path: /etc/systemd/system/containerd.service.d/http-proxy.conf
        permissions: 0644
        encoding: base64
        content: W1NlcnZpY2VdCkVudmlyb25tZW50PSJIVFRQX1BST1hZPWh0dHA6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0iSFRUUFNfUFJPWFk9aHR0cHM6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0iTk9fUFJPWFk9MTI3LjAuMC4xLGxvY2FsaG9zdCxzdmMsbG9jYWwsYXdlc29tZS5leGFtcGxlLmdpZ2FudGljLmlvLDE3Mi4zMS4wLjAvMTYsMTAwLjY0LjAuMC8xMixlbGIuYW1hem9uYXdzLmNvbSwxNjkuMjU0LjE2OS4yNTQsc29tZS5ub3Byb3h5LmF3ZXNvbWUuZXhhbXBsZS5naWdhbnRpYy5pbyxhbm90aGVyLm5vcHJveHkuYWRkcmVzcy5naWFudHN3YXJtLmlvLHByb3h5MS5leGFtcGxlLmNvbSxwcm94eTIuZXhhbXBsZS5jb20iCkVudmlyb25tZW50PSJodHRwX3Byb3h5PWh0dHA6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0iaHR0cHNfcHJveHk9aHR0cHM6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0ibm9fcHJveHk9MTI3LjAuMC4xLGxvY2FsaG9zdCxzdmMsbG9jYWwsYXdlc29tZS5leGFtcGxlLmdpZ2FudGljLmlvLDE3Mi4zMS4wLjAvMTYsMTAwLjY0LjAuMC8xMixlbGIuYW1hem9uYXdzLmNvbSwxNjkuMjU0LjE2OS4yNTQsc29tZS5ub3Byb3h5LmF3ZXNvbWUuZXhhbXBsZS5naWdhbnRpYy5pbyxhbm90aGVyLm5vcHJveHkuYWRkcmVzcy5naWFudHN3YXJtLmlvLHByb3h5MS5leGFtcGxlLmNvbSxwcm94eTIuZXhhbXBsZS5jb20iCg==
      - path: /etc/systemd/system/kubelet.service.d/http-proxy.conf
        permissions: 0644
        encoding: base64
        content: W1NlcnZpY2VdCkVudmlyb25tZW50PSJIVFRQX1BST1hZPWh0dHA6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0iSFRUUFNfUFJPWFk9aHR0cHM6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0iTk9fUFJPWFk9MTI3LjAuMC4xLGxvY2FsaG9zdCxzdmMsbG9jYWwsYXdlc29tZS5leGFtcGxlLmdpZ2FudGljLmlvLDE3Mi4zMS4wLjAvMTYsMTAwLjY0LjAuMC8xMixlbGIuYW1hem9uYXdzLmNvbSwxNjkuMjU0LjE2OS4yNTQsc29tZS5ub3Byb3h5LmF3ZXNvbWUuZXhhbXBsZS5naWdhbnRpYy5pbyxhbm90aGVyLm5vcHJveHkuYWRkcmVzcy5naWFudHN3YXJtLmlvLHByb3h5MS5leGFtcGxlLmNvbSxwcm94eTIuZXhhbXBsZS5jb20iCkVudmlyb25tZW50PSJodHRwX3Byb3h5PWh0dHA6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0iaHR0cHNfcHJveHk9aHR0cHM6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0ibm9fcHJveHk9MTI3LjAuMC4xLGxvY2FsaG9zdCxzdmMsbG9jYWwsYXdlc29tZS5leGFtcGxlLmdpZ2FudGljLmlvLDE3Mi4zMS4wLjAvMTYsMTAwLjY0LjAuMC8xMixlbGIuYW1hem9uYXdzLmNvbSwxNjkuMjU0LjE2OS4yNTQsc29tZS5ub3Byb3h5LmF3ZXNvbWUuZXhhbXBsZS5naWdhbnRpYy5pbyxhbm90aGVyLm5vcHJveHkuYWRkcmVzcy5naWFudHN3YXJtLmlvLHByb3h5MS5leGFtcGxlLmNvbSxwcm94eTIuZXhhbXBsZS5jb20iCg==
      - path: /etc/systemd/system/teleport.service.d/http-proxy.conf
        permissions: 0644
        encoding: base64
        content: W1NlcnZpY2VdCkVudmlyb25tZW50PSJIVFRQX1BST1hZPWh0dHA6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0iSFRUUFNfUFJPWFk9aHR0cHM6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0iTk9fUFJPWFk9MTI3LjAuMC4xLGxvY2FsaG9zdCxzdmMsbG9jYWwsYXdlc29tZS5leGFtcGxlLmdpZ2FudGljLmlvLDE3Mi4zMS4wLjAvMTYsMTAwLjY0LjAuMC8xMixlbGIuYW1hem9uYXdzLmNvbSwxNjkuMjU0LjE2OS4yNTQsc29tZS5ub3Byb3h5LmF3ZXNvbWUuZXhhbXBsZS5naWdhbnRpYy5pbyxhbm90aGVyLm5vcHJveHkuYWRkcmVzcy5naWFudHN3YXJtLmlvLHByb3h5MS5leGFtcGxlLmNvbSxwcm94eTIuZXhhbXBsZS5jb20iCkVudmlyb25tZW50PSJodHRwX3Byb3h5PWh0dHA6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0iaHR0cHNfcHJveHk9aHR0cHM6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0ibm9fcHJveHk9MTI3LjAuMC4xLGxvY2FsaG9zdCxzdmMsbG9jYWwsYXdlc29tZS5leGFtcGxlLmdpZ2FudGljLmlvLDE3Mi4zMS4wLjAvMTYsMTAwLjY0LjAuMC8xMixlbGIuYW1hem9uYXdzLmNvbSwxNjkuMjU0LjE2OS4yNTQsc29tZS5ub3Byb3h5LmF3ZXNvbWUuZXhhbXBsZS5naWdhbnRpYy5pbyxhbm90aGVyLm5vcHJveHkuYWRkcmVzcy5naWFudHN3YXJtLmlvLHByb3h5MS5leGFtcGxlLmNvbSxwcm94eTIuZXhhbXBsZS5jb20iCg==
      - path: /etc/teleport-join-token
        permissions: 0644
        contentFrom:
          secret:
            name: awesome-teleport-join-token
            key: joinToken
      - path: /opt/teleport-node-role.sh
        permissions: 0755
        encoding: base64
        content: IyEvYmluL2Jhc2gKCmlmIHN5c3RlbWN0bCBpcy1hY3RpdmUgLXEga3ViZWxldC5zZXJ2aWNlOyB0aGVuCiAgICBpZiBbIC1lICIvZXRjL2t1YmVybmV0ZXMvbWFuaWZlc3RzL2t1YmUtYXBpc2VydmVyLnlhbWwiIF07IHRoZW4KICAgICAgICBlY2hvICJjb250cm9sLXBsYW5lIgogICAgZWxzZQogICAgICAgIGVjaG8gIndvcmtlciIKICAgIGZpCmVsc2UKICAgIGVjaG8gIiIKZmkK
      - path: /etc/teleport.yaml
        permissions: 0644
        encoding: base64
        content: dmVyc2lvbjogdjMKdGVsZXBvcnQ6CiAgZGF0YV9kaXI6IC92YXIvbGliL3RlbGVwb3J0CiAgam9pbl9wYXJhbXM6CiAgICB0b2tlbl9uYW1lOiAvZXRjL3RlbGVwb3J0LWpvaW4tdG9rZW4KICAgIG1ldGhvZDogdG9rZW4KICBwcm94eV9zZXJ2ZXI6IHRlbGVwb3J0LmdpYW50c3dhcm0uaW86NDQzCiAgbG9nOgogICAgb3V0cHV0OiBzdGRlcnIKYXV0aF9zZXJ2aWNlOgogIGVuYWJsZWQ6ICJubyIKc3NoX3NlcnZpY2U6CiAgZW5hYmxlZDogInllcyIKICBjb21tYW5kczoKICAtIG5hbWU6IG5vZGUKICAgIGNvbW1hbmQ6IFtob3N0bmFtZV0KICAgIHBlcmlvZDogMjRoMG0wcwogIC0gbmFtZTogYXJjaAogICAgY29tbWFuZDogW3VuYW1lLCAtbV0KICAgIHBlcmlvZDogMjRoMG0wcwogIC0gbmFtZTogcm9sZQogICAgY29tbWFuZDogWy9vcHQvdGVsZXBvcnQtbm9kZS1yb2xlLnNoXQogICAgcGVyaW9kOiAxbTBzCiAgbGFiZWxzOgogICAgaW5zOiBnaWFudG1jCiAgICBtYzogZ2lhbnRtYwogICAgY2x1c3RlcjogYXdlc29tZQogICAgYmFzZURvbWFpbjogZXhhbXBsZS5naWdhbnRpYy5pbwpwcm94eV9zZXJ2aWNlOgogIGVuYWJsZWQ6ICJubyIK
      - path: /etc/audit/rules.d/99-default.rules
        permissions: 0640
        encoding: base64
        content: IyBPdmVycmlkZGVuIGJ5IEdpYW50IFN3YXJtLgotYSBleGl0LGFsd2F5cyAtRiBhcmNoPWI2NCAtUyBleGVjdmUgLWsgYXVkaXRpbmcKLWEgZXhpdCxhbHdheXMgLUYgYXJjaD1iMzIgLVMgZXhlY3ZlIC1rIGF1ZGl0aW5nCg==
      - contentFrom:
          secret:
            key: node-stuff
            name: cluster-super-secret
        path: /etc/aws/node/file.yaml
        permissions: 0644
      - contentFrom:
          secret:
            key: node-stuff
            name: cluster-super-secret
        path: /etc/custom/node/file.yaml
        permissions: 0644
      - contentFrom:
          secret:
            key: node-stuff
            name: cluster-super-secret-worker
        path: /etc/aws/worker/node/file.yaml
        permissions: 0644
      - contentFrom:
          secret:
            key: node-stuff
            name: cluster-super-secret-worker
        path: /etc/custom/worker/node/file.yaml
        permissions: 0644
    
  
    ---
    # Source: cluster/templates/clusterapi/workers/kubeadmconfig.yaml
    apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
    kind: KubeadmConfig
    metadata:
      name: awesome-def00-5b88d
      namespace: org-giantswarm
      annotations:
        machine-pool.giantswarm.io/name: awesome-def00
        important-cluster-value: 1000
        robots-need-this-in-the-cluster: eW91IGNhbm5vdCByZWFkIHRoaXMsIGJ1dCByb2JvdHMgY2FuCg==
        for-robots-in-nodepool: cm9ib3RzIGFyZSBvcGVyYXRpbmcgb24gdGhpcyBub2RlIHBvb2wK
      labels:
        # deprecated: "app: cluster-aws" label is deprecated and it will be removed after upgrading
    # to Kubernetes 1.25. We still need it here because existing ClusterResourceSet selectors
    # need this label on the Cluster resource.
    app: cluster-aws
        app.kubernetes.io/name: cluster
        app.kubernetes.io/version: 1.2.2
        app.kubernetes.io/part-of: cluster-aws
        app.kubernetes.io/instance: release-name
        app.kubernetes.io/managed-by: Helm
        helm.sh/chart: cluster-1.2.2
        application.giantswarm.io/team: turtles
        giantswarm.io/cluster: awesome
        giantswarm.io/organization: giantswarm
        giantswarm.io/service-priority: highest
        cluster.x-k8s.io/cluster-name: awesome
        cluster.x-k8s.io/watch-filter: capi
        release.giantswarm.io/version: 27.0.0-alpha.1
        another-cluster-label: label-2
        some-cluster-label: label-1
        giantswarm.io/machine-pool: awesome-def00
        nodepool-workload-type: ai
    spec:
      format: ignition
      ignition:
        containerLinuxConfig:
          additionalConfig: |
            systemd:
              units:      
              - name: os-hardening.service
                enabled: true
                contents: |
                  [Unit]
                  Description=Apply os hardening
                  [Service]
                  Type=oneshot
                  ExecStartPre=-/bin/bash -c "gpasswd -d core rkt; gpasswd -d core docker; gpasswd -d core wheel"
                  ExecStartPre=/bin/bash -c "until [ -f '/etc/sysctl.d/hardening.conf' ]; do echo Waiting for sysctl file; sleep 1s;done;"
                  ExecStart=/usr/sbin/sysctl -p /etc/sysctl.d/hardening.conf
                  [Install]
                  WantedBy=multi-user.target
              - name: update-engine.service
                enabled: false
                mask: true
              - name: locksmithd.service
                enabled: false
                mask: true
              - name: sshkeys.service
                enabled: false
                mask: true
              - name: teleport.service
                enabled: true
                contents: |
                  [Unit]
                  Description=Teleport Service
                  After=network.target
                  [Service]
                  Type=simple
                  Restart=on-failure
                  ExecStart=/opt/bin/teleport start --roles=node --config=/etc/teleport.yaml --pid-file=/run/teleport.pid
                  ExecReload=/bin/kill -HUP $MAINPID
                  PIDFile=/run/teleport.pid
                  LimitNOFILE=524288
                  [Install]
                  WantedBy=multi-user.target
              - name: kubeadm.service
                dropins:
                - name: 10-flatcar.conf
                  contents: |
                    [Unit]
                    # kubeadm must run after coreos-metadata populated /run/metadata directory.
                    Requires=coreos-metadata.service
                    After=coreos-metadata.service
                    # kubeadm must run after containerd - see https://github.com/kubernetes-sigs/image-builder/issues/939.
                    After=containerd.service
                    # kubeadm requires having an IP
                    After=network-online.target
                    Wants=network-online.target
                    [Service]
                    # Ensure kubeadm service has access to kubeadm binary in /opt/bin on Flatcar.
                    Environment=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/opt/bin
                    # To make metadata environment variables available for pre-kubeadm commands.
                    EnvironmentFile=/run/metadata/*
              - name: containerd.service
                enabled: true
                contents: |
                dropins:
                - name: 10-change-cgroup.conf
                  contents: |
                    [Service]
                    CPUAccounting=true
                    MemoryAccounting=true
                    Slice=kubereserved.slice
              - name: auditd.service
                enabled: false      
              - name: var-lib-kubelet.mount
                enabled: true
                mask: false
                contents: |
                  [Unit]
                  Description=kubelet volume
                  DefaultDependencies=no
                  [Mount]
                  What=/dev/disk/by-label/kubelet
                  Where=/var/lib/kubelet
                  Type=xfs
                  [Install]
                  WantedBy=local-fs-pre.target
              - name: var-lib-containerd.mount
                enabled: true
                mask: false
                contents: |
                  [Unit]
                  Description=containerd volume
                  DefaultDependencies=no
                  [Mount]
                  What=/dev/disk/by-label/containerd
                  Where=/var/lib/containerd
                  Type=xfs
                  [Install]
                  WantedBy=local-fs-pre.target
              - name: example2.service
                enabled: false
                mask: false
                dropins:
                - name: hello1.conf
                  contents: |
                    # Multi-line
                    # contents goes here
                - name: hello2.conf
                  contents: |
                    # Multi-line
                    # contents goes here
              - name: var-lib-workload.mount
                enabled: true
                mask: false
                contents: |
                  [Unit]
                  Description=workload volume
                  DefaultDependencies=no
                  [Mount]
                  What=/dev/disk/by-label/workload
                  Where=/var/lib/workload
                  Type=xfs
                  [Install]
                  WantedBy=local-fs-pre.target
              - name: example2-workers.service
                enabled: false
                mask: false
                dropins:
                - name: hello1-workers.conf
                  contents: |
                    # Multi-line
                    # contents goes here
                - name: hello2-workers.conf
                  contents: |
                    # Multi-line
                    # contents goes here
            storage:
              filesystems:      
              directories:      
              - path: /var/lib/kubelet
                mode: 0750      
              - path: /var/lib/kubelet/temporary/stuff
                overwrite: true
                filesystem: kubelet
                mode: 750
                user:
                  id: 12345
                  name: giantswarm
                group:
                  id: 23456
                  name: giantswarm
              - path: /var/lib/kubelet/temporary/stuff/workers
                overwrite: true
                filesystem: kubelet
                mode: 750
                user:
                  id: 12345
                  name: giantswarm
                group:
                  id: 23456
                  name: giantswarm
            
      joinConfiguration:
        nodeRegistration:
          name: ${COREOS_EC2_HOSTNAME}
          kubeletExtraArgs:
            cgroup-driver: cgroupfs
            cloud-provider: external
            healthz-bind-address: 0.0.0.0
            node-ip: ${COREOS_EC2_IPV4_LOCAL}
            node-labels: "ip=${COREOS_EC2_IPV4_LOCAL},role=worker,giantswarm.io/machine-pool=awesome-def00,workload-type=ai"
            v: 2
          taints:
          - key: supernodepool
            value: hello
            effect: NoSchedule
        patches:
          directory: /etc/kubernetes/patches
      preKubeadmCommands:
      - "envsubst < /etc/kubeadm.yml > /etc/kubeadm.yml.tmp"
      - "mv /etc/kubeadm.yml.tmp /etc/kubeadm.yml"
      - "systemctl restart containerd"
      - "export HTTP_PROXY=http://proxy.giantswarm.io"
      - "export HTTPS_PROXY=https://proxy.giantswarm.io"
      - "export NO_PROXY="127.0.0.1,localhost,svc,local,awesome.example.gigantic.io,172.31.0.0/16,100.64.0.0/12,elb.amazonaws.com,169.254.169.254,some.noproxy.awesome.example.gigantic.io,another.noproxy.address.giantswarm.io,proxy1.example.com,proxy2.example.com""
      - "export http_proxy=http://proxy.giantswarm.io"
      - "export https_proxy=https://proxy.giantswarm.io"
      - "export no_proxy="127.0.0.1,localhost,svc,local,awesome.example.gigantic.io,172.31.0.0/16,100.64.0.0/12,elb.amazonaws.com,169.254.169.254,some.noproxy.awesome.example.gigantic.io,another.noproxy.address.giantswarm.io,proxy1.example.com,proxy2.example.com""
      - "echo "aws nodes command before kubeadm""
      - "echo "custom nodes command before kubeadm""
      - "echo "aws workers command before kubeadm""
      - "echo "custom workers command before kubeadm""
      postKubeadmCommands:
      - "echo "aws nodes command after kubeadm""
      - "echo "custom nodes command after kubeadm""
      - "echo "aws workers command after kubeadm""
      - "echo "custom workers command after kubeadm""
      files:
      - path: /etc/sysctl.d/hardening.conf
        permissions: 0644
        encoding: base64
        content: ZnMuaW5vdGlmeS5tYXhfdXNlcl93YXRjaGVzID0gMTYzODQKZnMuaW5vdGlmeS5tYXhfdXNlcl9pbnN0YW5jZXMgPSA4MTkyCmtlcm5lbC5rcHRyX3Jlc3RyaWN0ID0gMgprZXJuZWwuc3lzcnEgPSAwCm5ldC5pcHY0LmNvbmYuYWxsLmxvZ19tYXJ0aWFucyA9IDEKbmV0LmlwdjQuY29uZi5hbGwuc2VuZF9yZWRpcmVjdHMgPSAwCm5ldC5pcHY0LmNvbmYuZGVmYXVsdC5hY2NlcHRfcmVkaXJlY3RzID0gMApuZXQuaXB2NC5jb25mLmRlZmF1bHQubG9nX21hcnRpYW5zID0gMQpuZXQuaXB2NC50Y3BfdGltZXN0YW1wcyA9IDAKbmV0LmlwdjYuY29uZi5hbGwuYWNjZXB0X3JlZGlyZWN0cyA9IDAKbmV0LmlwdjYuY29uZi5kZWZhdWx0LmFjY2VwdF9yZWRpcmVjdHMgPSAwCiMgSW5jcmVhc2VkIG1tYXBmcyBiZWNhdXNlIHNvbWUgYXBwbGljYXRpb25zLCBsaWtlIEVTLCBuZWVkIGhpZ2hlciBsaW1pdCB0byBzdG9yZSBkYXRhIHByb3Blcmx5CnZtLm1heF9tYXBfY291bnQgPSAyNjIxNDQKIyBSZXNlcnZlZCB0byBhdm9pZCBjb25mbGljdHMgd2l0aCBrdWJlLWFwaXNlcnZlciwgd2hpY2ggYWxsb2NhdGVzIHdpdGhpbiB0aGlzIHJhbmdlCm5ldC5pcHY0LmlwX2xvY2FsX3Jlc2VydmVkX3BvcnRzPTMwMDAwLTMyNzY3Cm5ldC5pcHY0LmNvbmYuYWxsLnJwX2ZpbHRlciA9IDEKbmV0LmlwdjQuY29uZi5hbGwuYXJwX2lnbm9yZSA9IDEKbmV0LmlwdjQuY29uZi5hbGwuYXJwX2Fubm91bmNlID0gMgoKIyBUaGVzZSBhcmUgcmVxdWlyZWQgZm9yIHRoZSBrdWJlbGV0ICctLXByb3RlY3Qta2VybmVsLWRlZmF1bHRzJyBmbGFnCiMgU2VlIGh0dHBzOi8vZ2l0aHViLmNvbS9naWFudHN3YXJtL2dpYW50c3dhcm0vaXNzdWVzLzEzNTg3CnZtLm92ZXJjb21taXRfbWVtb3J5PTEKa2VybmVsLnBhbmljPTEwCmtlcm5lbC5wYW5pY19vbl9vb3BzPTEK
      - path: /etc/selinux/config
        permissions: 0644
        encoding: base64
        content: IyBUaGlzIGZpbGUgY29udHJvbHMgdGhlIHN0YXRlIG9mIFNFTGludXggb24gdGhlIHN5c3RlbSBvbiBib290LgoKIyBTRUxJTlVYIGNhbiB0YWtlIG9uZSBvZiB0aGVzZSB0aHJlZSB2YWx1ZXM6CiMgICAgICAgZW5mb3JjaW5nIC0gU0VMaW51eCBzZWN1cml0eSBwb2xpY3kgaXMgZW5mb3JjZWQuCiMgICAgICAgcGVybWlzc2l2ZSAtIFNFTGludXggcHJpbnRzIHdhcm5pbmdzIGluc3RlYWQgb2YgZW5mb3JjaW5nLgojICAgICAgIGRpc2FibGVkIC0gTm8gU0VMaW51eCBwb2xpY3kgaXMgbG9hZGVkLgpTRUxJTlVYPXBlcm1pc3NpdmUKCiMgU0VMSU5VWFRZUEUgY2FuIHRha2Ugb25lIG9mIHRoZXNlIGZvdXIgdmFsdWVzOgojICAgICAgIHRhcmdldGVkIC0gT25seSB0YXJnZXRlZCBuZXR3b3JrIGRhZW1vbnMgYXJlIHByb3RlY3RlZC4KIyAgICAgICBzdHJpY3QgICAtIEZ1bGwgU0VMaW51eCBwcm90ZWN0aW9uLgojICAgICAgIG1scyAgICAgIC0gRnVsbCBTRUxpbnV4IHByb3RlY3Rpb24gd2l0aCBNdWx0aS1MZXZlbCBTZWN1cml0eQojICAgICAgIG1jcyAgICAgIC0gRnVsbCBTRUxpbnV4IHByb3RlY3Rpb24gd2l0aCBNdWx0aS1DYXRlZ29yeSBTZWN1cml0eQojICAgICAgICAgICAgICAgICAgKG1scywgYnV0IG9ubHkgb25lIHNlbnNpdGl2aXR5IGxldmVsKQpTRUxJTlVYVFlQRT1tY3MK
      - path: /etc/systemd/timesyncd.conf
        permissions: 0644
        encoding: base64
        content: W1RpbWVdCk5UUD0xNjkuMjU0LjE2OS4xMjMK
      - path: /etc/flatcar-cgroupv1
        filesystem: root
        permissions: 0444
      - path: /etc/containerd/config.toml
        permissions: 0644
        contentFrom:
          secret:
            name: awesome-containerd-dbb9d89e
            key: config.toml
      - path: /etc/kubernetes/patches/kubeletconfiguration.yaml
        permissions: 0644
        encoding: base64
        content: YXBpVmVyc2lvbjoga3ViZWxldC5jb25maWcuazhzLmlvL3YxYmV0YTEKa2luZDogS3ViZWxldENvbmZpZ3VyYXRpb24Kc2h1dGRvd25HcmFjZVBlcmlvZDogMzAwcwpzaHV0ZG93bkdyYWNlUGVyaW9kQ3JpdGljYWxQb2RzOiA2MHMKa2VybmVsTWVtY2dOb3RpZmljYXRpb246IHRydWUKZXZpY3Rpb25Tb2Z0OgogIG1lbW9yeS5hdmFpbGFibGU6ICI1MDBNaSIKZXZpY3Rpb25IYXJkOgogIG1lbW9yeS5hdmFpbGFibGU6ICIyMDBNaSIKICBpbWFnZWZzLmF2YWlsYWJsZTogIjE1JSIKZXZpY3Rpb25Tb2Z0R3JhY2VQZXJpb2Q6CiAgbWVtb3J5LmF2YWlsYWJsZTogIjVzIgpldmljdGlvbk1heFBvZEdyYWNlUGVyaW9kOiA2MAprdWJlUmVzZXJ2ZWQ6CiAgY3B1OiAzNTBtCiAgbWVtb3J5OiAzODRNaQogIGVwaGVtZXJhbC1zdG9yYWdlOiAxMDI0TWkKa3ViZVJlc2VydmVkQ2dyb3VwOiAva3ViZXJlc2VydmVkLnNsaWNlCnByb3RlY3RLZXJuZWxEZWZhdWx0czogdHJ1ZQpzeXN0ZW1SZXNlcnZlZDoKICBjcHU6IDI1MG0KICBtZW1vcnk6IDEyODBNaQpzeXN0ZW1SZXNlcnZlZENncm91cDogL3N5c3RlbS5zbGljZQp0bHNDaXBoZXJTdWl0ZXM6Ci0gVExTX0FFU18xMjhfR0NNX1NIQTI1NgotIFRMU19BRVNfMjU2X0dDTV9TSEEzODQKLSBUTFNfQ0hBQ0hBMjBfUE9MWTEzMDVfU0hBMjU2Ci0gVExTX0VDREhFX0VDRFNBX1dJVEhfQUVTXzEyOF9DQkNfU0hBCi0gVExTX0VDREhFX0VDRFNBX1dJVEhfQUVTXzEyOF9HQ01fU0hBMjU2Ci0gVExTX0VDREhFX0VDRFNBX1dJVEhfQUVTXzI1Nl9DQkNfU0hBCi0gVExTX0VDREhFX0VDRFNBX1dJVEhfQUVTXzI1Nl9HQ01fU0hBMzg0Ci0gVExTX0VDREhFX0VDRFNBX1dJVEhfQ0hBQ0hBMjBfUE9MWTEzMDUKLSBUTFNfRUNESEVfRUNEU0FfV0lUSF9DSEFDSEEyMF9QT0xZMTMwNV9TSEEyNTYKLSBUTFNfRUNESEVfUlNBX1dJVEhfQUVTXzEyOF9DQkNfU0hBCi0gVExTX0VDREhFX1JTQV9XSVRIX0FFU18xMjhfR0NNX1NIQTI1NgotIFRMU19FQ0RIRV9SU0FfV0lUSF9BRVNfMjU2X0NCQ19TSEEKLSBUTFNfRUNESEVfUlNBX1dJVEhfQUVTXzI1Nl9HQ01fU0hBMzg0Ci0gVExTX0VDREhFX1JTQV9XSVRIX0NIQUNIQTIwX1BPTFkxMzA1Ci0gVExTX0VDREhFX1JTQV9XSVRIX0NIQUNIQTIwX1BPTFkxMzA1X1NIQTI1NgotIFRMU19SU0FfV0lUSF9BRVNfMTI4X0NCQ19TSEEKLSBUTFNfUlNBX1dJVEhfQUVTXzEyOF9HQ01fU0hBMjU2Ci0gVExTX1JTQV9XSVRIX0FFU18yNTZfQ0JDX1NIQQotIFRMU19SU0FfV0lUSF9BRVNfMjU2X0dDTV9TSEEzODQKc2VyaWFsaXplSW1hZ2VQdWxsczogZmFsc2UKc3RyZWFtaW5nQ29ubmVjdGlvbklkbGVUaW1lb3V0OiAxaAphbGxvd2VkVW5zYWZlU3lzY3RsczoKLSAibmV0LioiCmNvbnRhaW5lckxvZ01heFNpemU6IDMwTWkKY29udGFpbmVyTG9nTWF4RmlsZXM6IDIK
      - path: /etc/systemd/logind.conf.d/zzz-kubelet-graceful-shutdown.conf
        permissions: 0700
        encoding: base64
        content: W0xvZ2luXQojIGRlbGF5CkluaGliaXREZWxheU1heFNlYz0zMDAK
      - path: /etc/systemd/system/containerd.service.d/http-proxy.conf
        permissions: 0644
        encoding: base64
        content: W1NlcnZpY2VdCkVudmlyb25tZW50PSJIVFRQX1BST1hZPWh0dHA6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0iSFRUUFNfUFJPWFk9aHR0cHM6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0iTk9fUFJPWFk9MTI3LjAuMC4xLGxvY2FsaG9zdCxzdmMsbG9jYWwsYXdlc29tZS5leGFtcGxlLmdpZ2FudGljLmlvLDE3Mi4zMS4wLjAvMTYsMTAwLjY0LjAuMC8xMixlbGIuYW1hem9uYXdzLmNvbSwxNjkuMjU0LjE2OS4yNTQsc29tZS5ub3Byb3h5LmF3ZXNvbWUuZXhhbXBsZS5naWdhbnRpYy5pbyxhbm90aGVyLm5vcHJveHkuYWRkcmVzcy5naWFudHN3YXJtLmlvLHByb3h5MS5leGFtcGxlLmNvbSxwcm94eTIuZXhhbXBsZS5jb20iCkVudmlyb25tZW50PSJodHRwX3Byb3h5PWh0dHA6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0iaHR0cHNfcHJveHk9aHR0cHM6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0ibm9fcHJveHk9MTI3LjAuMC4xLGxvY2FsaG9zdCxzdmMsbG9jYWwsYXdlc29tZS5leGFtcGxlLmdpZ2FudGljLmlvLDE3Mi4zMS4wLjAvMTYsMTAwLjY0LjAuMC8xMixlbGIuYW1hem9uYXdzLmNvbSwxNjkuMjU0LjE2OS4yNTQsc29tZS5ub3Byb3h5LmF3ZXNvbWUuZXhhbXBsZS5naWdhbnRpYy5pbyxhbm90aGVyLm5vcHJveHkuYWRkcmVzcy5naWFudHN3YXJtLmlvLHByb3h5MS5leGFtcGxlLmNvbSxwcm94eTIuZXhhbXBsZS5jb20iCg==
      - path: /etc/systemd/system/kubelet.service.d/http-proxy.conf
        permissions: 0644
        encoding: base64
        content: W1NlcnZpY2VdCkVudmlyb25tZW50PSJIVFRQX1BST1hZPWh0dHA6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0iSFRUUFNfUFJPWFk9aHR0cHM6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0iTk9fUFJPWFk9MTI3LjAuMC4xLGxvY2FsaG9zdCxzdmMsbG9jYWwsYXdlc29tZS5leGFtcGxlLmdpZ2FudGljLmlvLDE3Mi4zMS4wLjAvMTYsMTAwLjY0LjAuMC8xMixlbGIuYW1hem9uYXdzLmNvbSwxNjkuMjU0LjE2OS4yNTQsc29tZS5ub3Byb3h5LmF3ZXNvbWUuZXhhbXBsZS5naWdhbnRpYy5pbyxhbm90aGVyLm5vcHJveHkuYWRkcmVzcy5naWFudHN3YXJtLmlvLHByb3h5MS5leGFtcGxlLmNvbSxwcm94eTIuZXhhbXBsZS5jb20iCkVudmlyb25tZW50PSJodHRwX3Byb3h5PWh0dHA6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0iaHR0cHNfcHJveHk9aHR0cHM6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0ibm9fcHJveHk9MTI3LjAuMC4xLGxvY2FsaG9zdCxzdmMsbG9jYWwsYXdlc29tZS5leGFtcGxlLmdpZ2FudGljLmlvLDE3Mi4zMS4wLjAvMTYsMTAwLjY0LjAuMC8xMixlbGIuYW1hem9uYXdzLmNvbSwxNjkuMjU0LjE2OS4yNTQsc29tZS5ub3Byb3h5LmF3ZXNvbWUuZXhhbXBsZS5naWdhbnRpYy5pbyxhbm90aGVyLm5vcHJveHkuYWRkcmVzcy5naWFudHN3YXJtLmlvLHByb3h5MS5leGFtcGxlLmNvbSxwcm94eTIuZXhhbXBsZS5jb20iCg==
      - path: /etc/systemd/system/teleport.service.d/http-proxy.conf
        permissions: 0644
        encoding: base64
        content: W1NlcnZpY2VdCkVudmlyb25tZW50PSJIVFRQX1BST1hZPWh0dHA6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0iSFRUUFNfUFJPWFk9aHR0cHM6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0iTk9fUFJPWFk9MTI3LjAuMC4xLGxvY2FsaG9zdCxzdmMsbG9jYWwsYXdlc29tZS5leGFtcGxlLmdpZ2FudGljLmlvLDE3Mi4zMS4wLjAvMTYsMTAwLjY0LjAuMC8xMixlbGIuYW1hem9uYXdzLmNvbSwxNjkuMjU0LjE2OS4yNTQsc29tZS5ub3Byb3h5LmF3ZXNvbWUuZXhhbXBsZS5naWdhbnRpYy5pbyxhbm90aGVyLm5vcHJveHkuYWRkcmVzcy5naWFudHN3YXJtLmlvLHByb3h5MS5leGFtcGxlLmNvbSxwcm94eTIuZXhhbXBsZS5jb20iCkVudmlyb25tZW50PSJodHRwX3Byb3h5PWh0dHA6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0iaHR0cHNfcHJveHk9aHR0cHM6Ly9wcm94eS5naWFudHN3YXJtLmlvIgpFbnZpcm9ubWVudD0ibm9fcHJveHk9MTI3LjAuMC4xLGxvY2FsaG9zdCxzdmMsbG9jYWwsYXdlc29tZS5leGFtcGxlLmdpZ2FudGljLmlvLDE3Mi4zMS4wLjAvMTYsMTAwLjY0LjAuMC8xMixlbGIuYW1hem9uYXdzLmNvbSwxNjkuMjU0LjE2OS4yNTQsc29tZS5ub3Byb3h5LmF3ZXNvbWUuZXhhbXBsZS5naWdhbnRpYy5pbyxhbm90aGVyLm5vcHJveHkuYWRkcmVzcy5naWFudHN3YXJtLmlvLHByb3h5MS5leGFtcGxlLmNvbSxwcm94eTIuZXhhbXBsZS5jb20iCg==
      - path: /etc/teleport-join-token
        permissions: 0644
        contentFrom:
          secret:
            name: awesome-teleport-join-token
            key: joinToken
      - path: /opt/teleport-node-role.sh
        permissions: 0755
        encoding: base64
        content: IyEvYmluL2Jhc2gKCmlmIHN5c3RlbWN0bCBpcy1hY3RpdmUgLXEga3ViZWxldC5zZXJ2aWNlOyB0aGVuCiAgICBpZiBbIC1lICIvZXRjL2t1YmVybmV0ZXMvbWFuaWZlc3RzL2t1YmUtYXBpc2VydmVyLnlhbWwiIF07IHRoZW4KICAgICAgICBlY2hvICJjb250cm9sLXBsYW5lIgogICAgZWxzZQogICAgICAgIGVjaG8gIndvcmtlciIKICAgIGZpCmVsc2UKICAgIGVjaG8gIiIKZmkK
      - path: /etc/teleport.yaml
        permissions: 0644
        encoding: base64
        content: dmVyc2lvbjogdjMKdGVsZXBvcnQ6CiAgZGF0YV9kaXI6IC92YXIvbGliL3RlbGVwb3J0CiAgam9pbl9wYXJhbXM6CiAgICB0b2tlbl9uYW1lOiAvZXRjL3RlbGVwb3J0LWpvaW4tdG9rZW4KICAgIG1ldGhvZDogdG9rZW4KICBwcm94eV9zZXJ2ZXI6IHRlbGVwb3J0LmdpYW50c3dhcm0uaW86NDQzCiAgbG9nOgogICAgb3V0cHV0OiBzdGRlcnIKYXV0aF9zZXJ2aWNlOgogIGVuYWJsZWQ6ICJubyIKc3NoX3NlcnZpY2U6CiAgZW5hYmxlZDogInllcyIKICBjb21tYW5kczoKICAtIG5hbWU6IG5vZGUKICAgIGNvbW1hbmQ6IFtob3N0bmFtZV0KICAgIHBlcmlvZDogMjRoMG0wcwogIC0gbmFtZTogYXJjaAogICAgY29tbWFuZDogW3VuYW1lLCAtbV0KICAgIHBlcmlvZDogMjRoMG0wcwogIC0gbmFtZTogcm9sZQogICAgY29tbWFuZDogWy9vcHQvdGVsZXBvcnQtbm9kZS1yb2xlLnNoXQogICAgcGVyaW9kOiAxbTBzCiAgbGFiZWxzOgogICAgaW5zOiBnaWFudG1jCiAgICBtYzogZ2lhbnRtYwogICAgY2x1c3RlcjogYXdlc29tZQogICAgYmFzZURvbWFpbjogZXhhbXBsZS5naWdhbnRpYy5pbwpwcm94eV9zZXJ2aWNlOgogIGVuYWJsZWQ6ICJubyIK
      - contentFrom:
          secret:
            name: cluster-super-secret
            key: node-stuff
        path: /etc/aws/node/file.yaml
        permissions: 0644
      - contentFrom:
          secret:
            name: cluster-super-secret
            key: node-stuff
        path: /etc/custom/node/file.yaml
        permissions: 0644
      - contentFrom:
          secret:
            name: cluster-super-secret-worker
            key: node-stuff
        path: /etc/aws/worker/node/file.yaml
        permissions: 0644
      - contentFrom:
          secret:
            name: cluster-super-secret-worker
            key: node-stuff
        path: /etc/custom/worker/node/file.yaml
        permissions: 0644
    # Source: cluster/templates/clusterapi/workers/kubeadmconfig.yaml
    apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
    kind: KubeadmConfig
    metadata:
      annotations:
        machine-pool.giantswarm.io/name: awesome-verybignodepool-1234
        important-cluster-value: 1000
        robots-need-this-in-the-cluster: eW91IGNhbm5vdCByZWFkIHRoaXMsIGJ1dCByb2JvdHMgY2FuCg==
      labels:
        # deprecated: "app: cluster-aws" label is deprecated and it will be removed after upgrading
    # to Kubernetes 1.25. We still need it here because existing ClusterResourceSet selectors
    # need this label on the Cluster resource.
    app: cluster-aws
        app.kubernetes.io/name: cluster
        app.kubernetes.io/version: 1.2.2
        app.kubernetes.io/part-of: cluster-aws
        app.kubernetes.io/instance: release-name
        app.kubernetes.io/managed-by: Helm
        helm.sh/chart: cluster-1.2.2
        application.giantswarm.io/team: turtles
        giantswarm.io/cluster: awesome
        giantswarm.io/organization: giantswarm
        giantswarm.io/service-priority: highest
        cluster.x-k8s.io/cluster-name: awesome
        cluster.x-k8s.io/watch-filter: capi
        release.giantswarm.io/version: 27.0.0-alpha.1
        another-cluster-label: label-2
        some-cluster-label: label-1
        giantswarm.io/machine-pool: awesome-verybignodepool-1234
      name: awesome-verybignodepool-1234-75665
      namespace: org-giantswarm
    spec:
      format: ignition
      ignition:
        containerLinuxConfig:
          additionalConfig: |
            systemd:
              units:      
              - name: os-hardening.service
                enabled: true
                contents: |
                  [Unit]
                  Description=Apply os hardening
                  [Service]
                  Type=oneshot
                  ExecStartPre=-/bin/bash -c "gpasswd -d core rkt; gpasswd -d core docker; gpasswd -d core wheel"
                  ExecStartPre=/bin/bash -c "until [ -f '/etc/sysctl.d/hardening.conf' ]; do echo Waiting for sysctl file; sleep 1s;done;"
                  ExecStart=/usr/sbin/sysctl -p /etc/sysctl.d/hardening.conf
                  [Install]
                  WantedBy=multi-user.target
              - name: update-engine.service
                enabled: false
                mask: true
              - name: locksmithd.service
                enabled: false
                mask: true
              - name: sshkeys.service
                enabled: false
                mask: true
              - name: teleport.service
                enabled: true
                contents: |
                  [Unit]
                  Description=Teleport Service
                  After=network.target
                  [Service]
                  Type=simple
                  Restart=on-failure
                  ExecStart=/opt/bin/teleport start --roles=node --config=/etc/teleport.yaml --pid-file=/run/teleport.pid
                  ExecReload=/bin/kill -HUP $MAINPID
                  PIDF...*[Comment body truncated]*

fiunchinho · 2024-09-06T20:03:30Z

helm/cluster/README.md

@@ -205,6 +205,8 @@ Advanced configuration of components that are running on all nodes.

 | **Property** | **Description** | **More Details** |
 | :----------- | :-------------- | :--------------- |
+| `global.components.auditd` | **Auditd** - Enable Auditd service.|**Type:** `object`<br/>|
+| `global.components.auditd.enabled` | **Enabled** - Whether or not the Auditd service shall be enabled. When true, the Auditd service is enabled. When false, the Auditd service is disabled.|**Type:** `boolean`<br/>**Default:** `false`|


Just thinking out loud here: should we warn inhere about potential performance issues when enabling it?

toggle audit

b702f6b

taylorbot mentioned this pull request Sep 6, 2024

Test cluster chart PR #325 giantswarm/cluster-aws#816

Closed

njuettner added 2 commits September 6, 2024 08:17

toggle audit

76bb6f7

toggle audit

3048766

taylorbot mentioned this pull request Sep 6, 2024

Test cluster chart PR #325 giantswarm/cluster-aws#818

Closed

njuettner added 2 commits September 6, 2024 09:54

toggle audit

ef52d47

toggle audit

530fe5e

njuettner changed the title ~~Toggle audit~~ Toggle auditd Sep 6, 2024

njuettner marked this pull request as ready for review September 6, 2024 09:46

njuettner requested a review from a team as a code owner September 6, 2024 09:46

njuettner requested a review from a team September 6, 2024 09:47

AverageMarcus approved these changes Sep 6, 2024

View reviewed changes

njuettner added 2 commits September 6, 2024 11:50

toggle audit

befb775

tidy

260fb17

njuettner mentioned this pull request Sep 6, 2024

Slow node performance due to audit configuration giantswarm/roadmap#3669

Closed

Gacko approved these changes Sep 6, 2024

View reviewed changes

njuettner merged commit 0433bf3 into main Sep 6, 2024
16 checks passed

njuettner deleted the toggle-audit branch September 6, 2024 11:33

fiunchinho reviewed Sep 6, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Toggle auditd #325

Toggle auditd #325

njuettner commented Sep 6, 2024 •

edited

Loading

taylorbot commented Sep 6, 2024

taylorbot commented Sep 6, 2024

github-actions bot commented Sep 6, 2024

fiunchinho Sep 6, 2024

Toggle auditd #325

Toggle auditd #325

Conversation

njuettner commented Sep 6, 2024 • edited Loading

Should this change be mentioned in the release notes?

taylorbot commented Sep 6, 2024

taylorbot commented Sep 6, 2024

github-actions bot commented Sep 6, 2024

fiunchinho Sep 6, 2024

Choose a reason for hiding this comment

njuettner commented Sep 6, 2024 •

edited

Loading