Using existing secret

[Arvindh-Mojo]

Hi - can you provide some guidance on this ? I'm trying to install the lacework agent using helm chart and use existing secret.

it works when i put the access token value in the values.yaml file but i dont want to expose the secret in helm but have it pull from AWS secrets manager. I have configured the secret-csi-driver and secret-store. I also have SecretProviderClass which will create and mount the kubernetes secret for lacework agent.

I'm following this documentation --> https://docs.lacework.com/onboarding/deploy-on-kubernetes#specify-an-existing-secret

The secret should be created with name LaceworkAcessToken in the EKS cluster.
my values.yaml

laceworkConfig:
  accessToken:
    existingSecret:
      key: LaceworkAccessToken
      name: LaceworkAccessToken
  env: {{my-env}}
  kubernetesCluster: {{my-cluster-name}}
  serviceAccountName: lacework-sa

however i end up getting the below error

  Error: Failed to render chart: exit status 1: Error: values don't meet the specifications of the schema(s) in the following chart(s):
  lacework-agent:
  - laceworkConfig: accessToken is required

can you let me know about this issue ?

[Arvindh-Mojo]

okay.. i figure out the problem. The documentation isn't correct.

It took the values.yaml when the schema is set like

lacework-agent:
  laceworkConfig:
    accessToken:
      existingSecret:
        key: LaceworkAccessToken
        name: LaceworkAccessToken
    env: {{my-env}}
    kubernetesCluster: {{my-cluster-name}}
    serviceAccountName: lacework-sa

[cirego]

Hello @Arvindh-Mojo, glad to hear you were able to debug the issue yourself! If you don't mind my asking, what command are you using to install the Helm charts? Are you embedding the charts into another chart? Are you using something like ArgoCD to deploy?

From reading your comment, it seems like you're embedding the lacework-agent charts into another chart, which is why you needed the extra lacework-agent section.

[Arvindh-Mojo]

Hey @cirego thanks for reaching out. Actually, i'm using helmfile which points to a local chart that uses the lacework chart. Though my previous issue was basically blocking to install any release (which fixed by adding lacework-agent) i'm still trying to figure out how i can make your charts use existingsecrets.

my setup is that, secrets are in aws secrets manager and i'm using the csi-secrets-driver and secrets-store-aws to sync the asm to k8s secrets.

right now. my setup looks like this

values.yaml

lacework-agent:
  laceworkConfig:
    accessToken:
      existingSecret:
        key: lacework-access-token
        name: lacework-access-secret
    env: dev
    kubernetesCluster: my-eks-cluster
    serviceAccountName: lacework-sa

chart.yaml

apiVersion: v2
appVersion: "1.0"
description: Lacework Agent
home: https://www.lacework.com
icon: https://www.lacework.com/wp-content/uploads/2019/07/Lacework_Logo_color_2019.svg
keywords:
  - monitoring
  - security
  - run-time
  - metric
  - troubleshooting
kubeVersion: '> 1.9.0-0'
maintainers:
  - email: [email protected]
    name: lacework-support
name: lacework-agent
version: 6.1.2
dependencies:
  - name: lacework-agent
    version: 6.1.2
    repository: https://lacework.github.io/helm-charts/

in my local charts/template im creating two crds - for a service account and secretproviderclass to sync the secrets.
if i make a daemonset.yaml under templates, copy yours and just update the volumemounts to point to existingsecrets, it doesnt work.

do you have any documentation around this i can reference ?

[cirego]

Hi @Arvindh-Mojo, the documentation that you linked is our existing documentation on how to use existing secrets with Kubernetes. This is also how our internal tests are configured to verify that this functionality works.

One thing you might consider would be having two charts: one that creates the existing secret and service account and then use the stock lacework agent charts with references to those resources.

Also, to confirm, if you run kubectl describe secrets lacework-access-secret, do you see a field called lacework-access-token under the data section?

[Arvindh-Mojo]

Yes, followed that documentation to configure stuffs but i guess my problem is with the daemonset.yaml
here -

helm-charts/lacework-agent/templates/daemonset.yaml

Line 57 in c8a991e

- name: LaceworkAccessToken

The secrets store csi driver communicates with the provider using gPRC to retrieve the secret content from the external secret store using SecretProviderClass and then volume is mounted in the pod. Basically, this tells me (and i verified with a sample deployment.yaml) that k8s secrets are created only when the pod comes up.

https://secrets-store-csi-driver.sigs.k8s.io/getting-started/usage.html#secret-content-is-mounted-on-pod-start

In a nutshell, i need the daemon set to come up properly. For a classic case where, i already have a k8s secret it would work but not in this case. something is off with the daemonset.yaml which i'm trying to figure out.

[cirego]

@Arvindh-Mojo, did you leave the daemonset as using SecretKeyRef or did you follow these instructions for using a file as a secret?

[Arvindh-Mojo]

@cirego i left the daemonset to use the SecretKeyRef. The problem that i have is, if i create a daemonset.yaml in my local chart to override your daemonset.yaml, the values pointer becomes in valid.

values.yaml

lacework-agent:
  laceworkConfig:
    accessToken:
      existingSecret:
        key: lacework_access_token
        name: lacework-access-secret
    env: dev
    kubernetesCluster: my-eks-cluster
    serviceAccountName: lacework-sa

however, in the daemonset.yaml (using the exact file) for example {{- with .Values.laceworkConfig.annotations }} becomes invalid and if i change it to {{- with .Values.lacework-agent.laceworkConfig.annotations }}, it wont work because the name is lacework-agentwith-` on it.

your daemonset.yaml snippet

apiVersion: apps/v1

kind: DaemonSet
metadata:
  name: {{ include "lacework-agent.name" . }}
  namespace: {{ .Release.Namespace }}
  labels:
    tier: monitoring
    app: {{ include "lacework-agent.name" . }}
    {{- with .Values.laceworkConfig.labels }}
      {{- toYaml . | nindent 4 }}
    {{- end }}
  {{- with .Values.laceworkConfig.annotations }}
  annotations:
    {{- toYaml . | nindent 8 }}
  {{- end }}

any suggestion of how to go about it. because i need to add volumes and volumeMounts in the daemonset.yaml to successfully sync and create secrets.

          volumeMounts:
            - name: secrets-store-inline
              mountPath: /mnt/secret-store
              readOnly: true
      volumes:
        - name: secrets-store-inline
          csi:
            driver: secrets-store.csi.k8s.io
            readOnly: true
            volumeAttributes:
              secretProviderClass: lacework-credentials

[Arvindh-Mojo]

@cirego Chris - we need to have some sort of way to enable/disabled the default daemonset.yaml to create our own yaml or way to add custom mount volumes using values.yaml. Later would be much easier because when using csi driver based secrets, we should allow the container spec to enable volume mounts.

[Arvindh-Mojo]

basically, these charts are incompatible when using an external secret store(in my case aws csi driver for secret store) which uses secrets-store.csi.k8s.io . The existingSecret approach only works when the secret is already present or you manually create it.

i made a local copy of your chart and modified the daemonset.yaml to include volume & mount for secrets and made another chart with this new local chart as its dependency. It works fine.

[cirego]

Hi @Arvindh-Mojo, if you can raise a support ticket, we can prioritize making some changes to the Helm Charts to make this work with secrets-store.csi.k8s.io. Thank you!

[Arvindh-Mojo]

@cirego thanks !! will do and I will probably do a PR. In order to have this work, all we need to have is the ability to mount additional volumes to the daemonset.yaml

[josh-hong-pps]

I'm currently seeing a similar issue. I'm using ArgoCD to deploy lacework agents using Helm Charts and I don't see a way to use kubernetes secrets to retrieve the accessToken. I've tried setting laceworkConfig.accessToken.existingSecret.name and the key as well to no avail while hardcoding the accessToken works.

I would think that would require a way to create a service account with similar permissions as well to access the kubernetes secrets too.

Any guidance on this or suggestions on current workarounds?

[josh-hong-pps]

After playing around with this a bit further, I think the issue comes down to how secrets are being retrieved in the ConfigMap. I've set laceworkConfig.accessToken.existingSecret.name and laceworkConfig.accessToken.existingSecret.key. I can see that the resulting ConfigMap looks like this:

apiVersion: v1
kind: ConfigMap
metadata:
  name: lacework-config
  namespace: lacework          
  labels:
    tier: monitoring
    app: lacework-agent
data:
  config.json: |
    {"autoupgrade": "enable",
     "ebpf": {
     },
     "fim": {
             "mode": "enable"
     },
     "serverurl": "https://api.lacework.net",
     "tokens":{"AccessToken": "map[existingSecret:map[key:accessToken name:lacework-agent-access-token]]"},
     "logstdout" : "true",
    }