Skip to content

Commit

Permalink
Merge pull request #211 from bank-vaults/chore/match-annotations-with…
Browse files Browse the repository at this point in the history
…-secrets_webhook-with-backwards-compatibility

chore: match annotations with secrets webhook
  • Loading branch information
akijakya authored Jun 11, 2024
2 parents f5e9f06 + e757412 commit bd03356
Show file tree
Hide file tree
Showing 10 changed files with 58 additions and 25 deletions.
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@ Vault Secrets Reloader can periodically check if a secret that is used in watche

Upon deployment, the Reloader spawns two “workers”, that run periodically at two different time intervals:

1. The `collector` collects and stores information about the workloads that are opted in via the `alpha.vault.security.banzaicloud.io/reload-on-secret-change: "true"` annotation in their pod template metadata and the Vault secrets they use.
1. The `collector` collects and stores information about the workloads that are opted in via the `secrets-reloader.security.bank-vaults.io/reload-on-secret-change: "true"` annotation in their pod template metadata and the Vault secrets they use.

2. The `reloader` iterates on the data collected by the `collector`, polling the configured Vault instance for the current version of the secrets, and if it finds that it differs from the stored one, adds the workloads where the secret is used to a list of workloads that needs reloading. In a following step, it modifies these workloads by incrementing the value of the `alpha.vault.security.banzaicloud.io/secret-reload-count` annotation in their pod template metadata, initiating a new rollout.
2. The `reloader` iterates on the data collected by the `collector`, polling the configured Vault instance for the current version of the secrets, and if it finds that it differs from the stored one, adds the workloads where the secret is used to a list of workloads that needs reloading. In a following step, it modifies these workloads by incrementing the value of the `secrets-reloader.security.bank-vaults.io/secret-reload-count` annotation in their pod template metadata, initiating a new rollout.

To get familiarized, check out [how Reloader fits in the Bank-Vaults ecosystem](https://github.com/bank-vaults/vault-secrets-reloader/blob/main/examples/reloader-in-bank-vaults-ecosystem.md), and how can you [give Reloader a spin](https://github.com/bank-vaults/vault-secrets-reloader/blob/main/examples/try-locally.md) on your local machine.

Expand All @@ -29,7 +29,7 @@ To get familiarized, check out [how Reloader fits in the Bank-Vaults ecosystem](

- It can only check for updated versions of secrets in one specific instance of Hashicorp Vault, no other secret stores are supported yet.

- It can only “reload” Deployments, DaemonSets and StatefulSets that have the `alpha.vault.security.banzaicloud.io/reload-on-secret-change: "true"` annotation set among their `spec.template.metadata.annotations`.
- It can only “reload” Deployments, DaemonSets and StatefulSets that have the `secrets-reloader.security.bank-vaults.io/reload-on-secret-change: "true"` annotation set among their `spec.template.metadata.annotations`.

- The `collector` can only look for secrets in the workload’s pod template environment variables directly, and in their `secrets-webhook.security.bank-vaults.io/vault-from-path` annotation, in the format the `secrets-webhook` also uses, and are unversioned.

Expand Down
2 changes: 1 addition & 1 deletion deploy/charts/vault-secrets-reloader/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Reloader works in conjunction with the [Secrets Webhook](https://github.com/bank
You will need to add the following annotations to the pod template spec of the workloads (i.e. Deployments, DaemonSets and StatefulSets) that you wish to reload:

```yaml
alpha.vault.security.banzaicloud.io/reload-on-secret-change: "true"
secrets-reloader.security.bank-vaults.io/reload-on-secret-change: "true"
```
## Installing the Chart
Expand Down
2 changes: 1 addition & 1 deletion deploy/charts/vault-secrets-reloader/README.md.gotmpl
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Reloader works in conjunction with the [Secrets Webhook](https://github.com/bank
You will need to add the following annotations to the pod template spec of the workloads (i.e. Deployments, DaemonSets and StatefulSets) that you wish to reload:

```yaml
alpha.vault.security.banzaicloud.io/reload-on-secret-change: "true"
secrets-reloader.security.bank-vaults.io/reload-on-secret-change: "true"
```

## Installing the Chart
Expand Down
2 changes: 1 addition & 1 deletion e2e/deploy/workloads/daemonset.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ spec:
secrets-webhook.security.bank-vaults.io/provider: "vault"
secrets-webhook.security.bank-vaults.io/vault-addr: "https://vault:8200"
secrets-webhook.security.bank-vaults.io/vault-tls-secret: vault-tls
alpha.vault.security.banzaicloud.io/reload-on-secret-change: "true"
secrets-reloader.security.bank-vaults.io/reload-on-secret-change: "true"
spec:
initContainers:
- name: init-ubuntu
Expand Down
12 changes: 8 additions & 4 deletions e2e/deploy/workloads/deployments.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,10 @@ spec:
labels:
app.kubernetes.io/name: reloader-test-deployment-to-be-reloaded
annotations:
secrets-webhook.security.bank-vaults.io/provider: "vault"
secrets-webhook.security.bank-vaults.io/vault-addr: "https://vault:8200"
secrets-webhook.security.bank-vaults.io/vault-tls-secret: vault-tls
alpha.vault.security.banzaicloud.io/reload-on-secret-change: "true"
secrets-reloader.security.bank-vaults.io/reload-on-secret-change: "true"
spec:
initContainers:
- name: init-ubuntu
Expand Down Expand Up @@ -62,6 +63,7 @@ spec:
labels:
app.kubernetes.io/name: reloader-test-deployment-no-reload
annotations:
secrets-webhook.security.bank-vaults.io/provider: "vault"
secrets-webhook.security.bank-vaults.io/vault-addr: "https://vault:8200"
secrets-webhook.security.bank-vaults.io/vault-tls-secret: vault-tls
spec:
Expand Down Expand Up @@ -98,9 +100,10 @@ spec:
labels:
app.kubernetes.io/name: reloader-test-deployment-fixed-versions-no-reload
annotations:
secrets-webhook.security.bank-vaults.io/provider: "vault"
secrets-webhook.security.bank-vaults.io/vault-addr: "https://vault:8200"
secrets-webhook.security.bank-vaults.io/vault-tls-secret: vault-tls
alpha.vault.security.banzaicloud.io/reload-on-secret-change: "true"
secrets-reloader.security.bank-vaults.io/reload-on-secret-change: "true"
spec:
containers:
- name: alpine
Expand Down Expand Up @@ -135,10 +138,11 @@ spec:
labels:
app.kubernetes.io/name: reloader-test-deployment-annotated-reload
annotations:
secrets-webhook.security.bank-vaults.io/provider: "vault"
secrets-webhook.security.bank-vaults.io/vault-addr: "https://vault:8200"
secrets-webhook.security.bank-vaults.io/vault-tls-secret: vault-tls
secrets-webhook.security.bank-vaults.io/vault-from-path: "secret/data/accounts/aws"
alpha.vault.security.banzaicloud.io/reload-on-secret-change: "true"
secrets-reloader.security.bank-vaults.io/reload-on-secret-change: "true"
spec:
containers:
- name: alpine
Expand Down Expand Up @@ -177,7 +181,7 @@ spec:
secrets-webhook.security.bank-vaults.io/vault-addr: "https://vault:8200"
secrets-webhook.security.bank-vaults.io/vault-tls-secret: vault-tls
secrets-webhook.security.bank-vaults.io/vault-from-path: "secret/data/dockerrepo#1"
alpha.vault.security.banzaicloud.io/reload-on-secret-change: "true"
secrets-reloader.security.bank-vaults.io/reload-on-secret-change: "true"
spec:
containers:
- name: alpine
Expand Down
2 changes: 1 addition & 1 deletion e2e/deploy/workloads/statefulset.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ spec:
secrets-webhook.security.bank-vaults.io/provider: "vault"
secrets-webhook.security.bank-vaults.io/vault-addr: "https://vault:8200"
secrets-webhook.security.bank-vaults.io/vault-tls-secret: vault-tls
alpha.vault.security.banzaicloud.io/reload-on-secret-change: "true"
secrets-reloader.security.bank-vaults.io/reload-on-secret-change: "true"
spec:
initContainers:
- name: init-ubuntu
Expand Down
2 changes: 1 addition & 1 deletion examples/reloader-in-bank-vaults-ecosystem.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ This is a high level overview of how the Reloader plays along with other compone

![flowchart](./assets/flowchart.png)

1. The `collector` worker periodically collects unversioned secrets from workloads with the `alpha.vault.security.banzaicloud.io/reload-on-secret-change: "true"` annotation that are in the format for injection by the Webhook.
1. The `collector` worker periodically collects unversioned secrets from workloads with the `secrets-reloader.security.bank-vaults.io/reload-on-secret-change: "true"` annotation that are in the format for injection by the Webhook.

2. At its scheduled time, the `reloader` worker checks in Vault if there is a new version of any of the collected secrets since the last sync. If it is the case, it continues to step 3, otherwise the workflow stops.

Expand Down
9 changes: 5 additions & 4 deletions examples/try-locally.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ Now that we have the Bank-Vaults ecosystem running in our kind cluster, we can d
kubectl apply -f e2e/deploy/workloads
```

Looking at the manifest of one of the deployments, the only difference from one that is prepared to work with the Bank-Vaults Webhook with all the annotations starting with `secrets-webhook.security.bank-vaults.io` and the env values starting with `vault:` is the presence of the new `alpha.vault.security.banzaicloud.io/reload-on-secret-change: "true"` annotation telling the Reloader to collect secrets and reload it if necessary.
Looking at the manifest of one of the deployments, the only difference from the one that is prepared to work with the Bank-Vaults Webhook with all the annotations starting with `secrets-webhook.security.bank-vaults.io` and the env values starting with `vault:` is the presence of the new `secrets-reloader.security.bank-vaults.io/reload-on-secret-change: "true"` annotation telling the Reloader to collect secrets and reload it if necessary.

```yaml
apiVersion: apps/v1
Expand All @@ -86,9 +86,10 @@ spec:
labels:
app.kubernetes.io/name: reloader-test-deployment-to-be-reloaded
annotations:
secrets-webhook.security.bank-vaults.io/provider: "vault"
secrets-webhook.security.bank-vaults.io/vault-addr: "https://vault:8200"
secrets-webhook.security.bank-vaults.io/vault-tls-secret: vault-tls
alpha.vault.security.banzaicloud.io/reload-on-secret-change: "true"
secrets-reloader.security.bank-vaults.io/reload-on-secret-change: "true"
spec:
initContainers:
- name: init-ubuntu
Expand Down Expand Up @@ -140,10 +141,10 @@ Now everything is set to try some things out with the Reloader:

Also notice that there are two pods with the now changed `MYSQL_PASSWORD` injected into them not being restarted, for the following reasons:

- the pod `reloader-test-deployment-no-reload-xxx` does not have the `alpha.vault.security.banzaicloud.io/reload-on-secret-change: "true"` annotation set
- the pod `reloader-test-deployment-no-reload-xxx` does not have the `secrets-reloader.security.bank-vaults.io/reload-on-secret-change: "true"` annotation set
- the pod `reloader-test-deployment-fixed-versions-no-reload-xxx` - although it does have the annotation - only uses versioned secrets, so they won't be reloaded for the latest version of the secret.
2. Change two secrets used in a workload, observe the previous pod to be recreated again, also that the pod `reloader-test-daemonset-xxx` only restarted once, although it uses both of these secrets. The number a workload got "reloaded" by the Reloader can be checked on the `alpha.vault.security.banzaicloud.io/secret-reload-count` annotation that is used to trigger a new rollout.
2. Change two secrets used in a workload, observe the previous pod to be recreated again, also that the pod `reloader-test-daemonset-xxx` only restarted once, although it uses both of these secrets. The number a workload got "reloaded" by the Reloader can be checked on the `secrets-reloader.security.bank-vaults.io/secret-reload-count` annotation that is used to trigger a new rollout.
```bash
vault kv patch secret/accounts/aws AWS_SECRET_ACCESS_KEY=s3cr3t2
Expand Down
4 changes: 2 additions & 2 deletions pkg/reloader/controller.go
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,8 @@ const (
DaemonSetKind = "DaemonSet"
StatefulSetKind = "StatefulSet"

SecretReloadAnnotationName = "alpha.vault.security.banzaicloud.io/reload-on-secret-change"
ReloadCountAnnotationName = "alpha.vault.security.banzaicloud.io/secret-reload-count"
SecretReloadAnnotationName = "secrets-reloader.security.bank-vaults.io/reload-on-secret-change"
ReloadCountAnnotationName = "secrets-reloader.security.bank-vaults.io/secret-reload-count"
)

// Controller is the controller implementation for Foo resources
Expand Down
42 changes: 35 additions & 7 deletions pkg/reloader/reloader_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,14 +23,42 @@ import (
)

func TestIncrementReloadCountAnnotation(t *testing.T) {
podTemplate := corev1.PodTemplateSpec{
ObjectMeta: metav1.ObjectMeta{
Annotations: map[string]string{},
tests := []struct {
name string
annotations map[string]string
expectedAnnotations map[string]string
}{
{
name: "no annotation should add annotation",
annotations: map[string]string{},
expectedAnnotations: map[string]string{
ReloadCountAnnotationName: "1",
},
},
{
name: "existing annotation should increment annotation",
annotations: map[string]string{
ReloadCountAnnotationName: "1",
},
expectedAnnotations: map[string]string{
ReloadCountAnnotationName: "2",
},
},
}

incrementReloadCountAnnotation(&podTemplate)
assert.Equal(t, "1", podTemplate.GetAnnotations()[ReloadCountAnnotationName])
incrementReloadCountAnnotation(&podTemplate)
assert.Equal(t, "2", podTemplate.GetAnnotations()[ReloadCountAnnotationName])
for _, tt := range tests {
ttp := tt
t.Run(ttp.name, func(t *testing.T) {

podTemplateSpec := &corev1.PodTemplateSpec{
ObjectMeta: metav1.ObjectMeta{
Annotations: ttp.annotations,
},
}

incrementReloadCountAnnotation(podTemplateSpec)

assert.Equal(t, ttp.expectedAnnotations, podTemplateSpec.Annotations)
})
}
}

0 comments on commit bd03356

Please sign in to comment.