Guide: How to prevent secrets committed in the repo

[kingdonb]

• validate.sh error "sops: Additional property sops is not allowed" flux2-kustomize-helm-example#48

In the issue above, we found that kubeconform can be told to ignore secrets since we properly encrypt them, they will not validate.

In #1481, I mentioned that users might combine IAM authorization to prevent unauthorized access and SOPS to encrypt the data and protect it from reading in case of unintentional disclosure. All of that goes out the window if users forget to encrypt the secret.

I have gone looking for a good answer to this question several times in the past, and I never found a really good one. I looked at pre-commit, but that depends on the users installing it. I think that potentially you can have a receive hook on a git repo that checks for secrets, but as soon as it failed and prevented work from going in for any unrelated reason I'd be ripping that thing back out and cursing under my breath. So the next-best solution would have been a scanner to alert us when a repository secret gets committed and pushed unencrypted, and take immediate action based on whenever that happens.

"It happens" so say we can't prevent it from happening, but we can have procedures in place for invalidating and rotating secrets when they may have been compromised, (and if it's possible to identify which secret source that secret belongs to, we can invalidate them automatically when they are first detected. Maybe some platform offers all that... it's all out of scope for Flux, but we are also the best positioned org to help tell folks about a solution for this problem space, if there's one that exists.)

Would this type of guide make sense to add somewhere in the Flux docs, or can we develop it separately and introduce it here later (in the github-app-secret project aegis?)

[kingdonb]

I tried to add pre-commit hooks but I found this approach was fraught (when I tried some time ago), and I never tried again so far.

Maybe it would be better if I built my pre-commit hooks using kubeconform itself? If you do want to allow secrets in the repo, but only encrypted ones, perhaps we can propose an enhancement to kubeconform itself, to present a way for users to prevent accidentally committing an unencrypted secret with its data exposed.

I think it's true that using Flux with SOPS demands or implies that secrets will be stored in the repo in one way or another. Whether that repo is a separate one, with different access controls, is for policy to determine. The more we discuss storing secrets in a repo at all, the more attention this topic needs.

Allowing secrets through the validator only solves half the problem, kubeconform in a precommit hook would be more ideal if it could also alert on non-encrypted secrets that are valid (don't only skip, but validate there are not any plain-text secrets!)

Is this an anti-pattern or can we offer prescriptive guidance about how and why to do it this way?

https://github.com/yuvipanda/pre-commit-hook-ensure-sops

I found this, it looks familiar, this might be the pre-commit hook I tried using before that failed. It looks like it has been updated more recently than I tried using it, though, so maybe it has been improved since then. I'm still wary because of bad experiences before, and I'm not that familiar with Python so maybe someone who is can be a better judge if this is any good.