Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(providers): Add support for fetching from k8s secrets #186

Merged
merged 7 commits into from
Dec 14, 2023
Merged

feat(providers): Add support for fetching from k8s secrets #186

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
5d7d54e
feat(providers): Add support for fetching from k8s secrets
bersalazar Dec 11, 2023
ea63792
fix(lint): fix linting and error handling
bersalazar Dec 12, 2023
4616e3d
chore(test): add a test, improve error handling
bersalazar Dec 12, 2023
c0f7f64
feat(k8s): update secret API reference
bersalazar Dec 13, 2023
a1dff61
fix(lint): minor lint and doc update
bersalazar Dec 14, 2023
7b6a295
fix: return values for kubeConfigPath and env var
bersalazar Dec 14, 2023
003edf4
feat(k8s): more test coverage
bersalazar Dec 14, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 24 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ It supports various backends including:
- [Doppler](https://doppler.com/)
- CredHub(Coming soon)
- Pulumi State
- Kubernetes secrets

- Use `vals eval -f refs.yaml` to replace all the `ref`s in the file to actual values and secrets.
- Use `vals exec -f env.yaml -- <COMMAND>` to populate envvars and execute the command.
Expand Down Expand Up @@ -190,7 +191,7 @@ EOF

`FRAGMENT` is a path-like expression that is used to extract a single value within the secret. When a fragment is specified, `vals` parse the secret value denoted by the `PATH` into a YAML or JSON object, and traverses the object following the fragment, and uses the value at the path as the final secret value. It's supposed to be the "fragment" componet of the URI as defined in [RFC3986](https://www.rfc-editor.org/rfc/rfc3986).

Finally, the optional trailing `+` is the explit "end" of the expression. You usually don't need it, as if omitted, it treats anything after `ref+` and before the new-line or the end-of-line as an expression to be evaluated. An explicit `+` is handy when you want to do a simple string interpolation. That is, `foo ref+SECRET1+ ref+SECRET2+ bar` evaluates to `foo SECRET1_VALUE SECRET2_VALUE bar`.
Finally, the optional trailing `+` is the explicit "end" of the expression. You usually don't need it, as if omitted, it treats anything after `ref+` and before the new-line or the end-of-line as an expression to be evaluated. An explicit `+` is handy when you want to do a simple string interpolation. That is, `foo ref+SECRET1+ ref+SECRET2+ bar` evaluates to `foo SECRET1_VALUE SECRET2_VALUE bar`.

Although we mention the RFC for the sake of explanation, `PARAMS` and `FRAGMENT` might not be fully RFC-compliant as, under the hood, we use a simple regexp that seemed to work for most of use-cases.

Expand Down Expand Up @@ -218,6 +219,7 @@ Please see the [relevant unit test cases](https://github.com/helmfile/vals/blob/
- [1Password Connect](#1password-connect)
- [Doppler](#doppler)
- [Pulumi State](#pulumi-state)
- [Kubernetes secrets](#kubernetes-secrets)

Please see [pkg/providers](https://github.com/helmfile/vals/tree/master/pkg/providers) for the implementations of all the providers. The package names corresponds to the URI schemes.

Expand Down Expand Up @@ -722,6 +724,27 @@ Examples:
- `ref+pulumistateapi://aws-native_s3_Bucket/my-bucket/outputs/tags.%23(key==SomeKey).value?project=my-project&stack=my-stack`
- `ref+pulumistateapi://kubernetes_storage.k8s.io__v1_StorageClass/gp2-encrypted/inputs/metadata.name?project=my-project&stack=my-stack`

### Kubernetes secrets

Fetch values from Kubernetes secrets:

- `ref+k8s://NAMESPACE/SECRET_NAME/KEY[?kubeConfigPath=<path_to_kubeconfig>&kubeContext=<kubernetes context name>]`

Authentication to the Kubernetes cluster is done by referencing the local kubeconfig file.
The path to the kubeconfig can be specified as a URI parameter, read from the `KUBECONFIG` environment variable or the provider with attempt to read `$HOME/.kube/config`.
The Kubernetes context can be specified as a URI parameteter.

Environment variables:

- `KUBECONFIG` contains the path to the Kubeconfig that will be used to fetch the secret.

Examples:

- `ref+k8s://mynamespace/mysecret/foo`
- `ref+k8s://mynamespace/mysecret/bar?kubeConfigPath=/home/user/kubeconfig`
- `secretref+k8s://mynamespace/secrets/baz`
- `secretref+k8s://mynamespace/secrets/baz?kubeContext=minikube`

## Advanced Usages

### Discriminating config and secrets
Expand Down
30 changes: 28 additions & 2 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,12 @@ require (
github.com/hashicorp/golang-lru v0.5.4
github.com/hashicorp/vault/api v1.10.0
github.com/stretchr/testify v1.8.4
github.com/tidwall/gjson v1.17.0
golang.org/x/oauth2 v0.12.0
google.golang.org/api v0.141.0
gopkg.in/yaml.v3 v3.0.1
k8s.io/apimachinery v0.28.4
k8s.io/client-go v0.28.4
)

require (
Expand Down Expand Up @@ -73,14 +76,22 @@ require (
github.com/cpuguy83/go-md2man/v2 v2.0.2 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/dimchansky/utfbom v1.1.1 // indirect
github.com/emicklei/go-restful/v3 v3.9.0 // indirect
github.com/fatih/color v1.15.0 // indirect
github.com/getsops/gopgagent v0.0.0-20170926210634-4d7ea76ff71a // indirect
github.com/go-jose/go-jose/v3 v3.0.1 // indirect
github.com/go-logr/logr v1.2.4 // indirect
github.com/go-openapi/jsonpointer v0.19.6 // indirect
github.com/go-openapi/jsonreference v0.20.2 // indirect
github.com/go-openapi/swag v0.22.3 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang-jwt/jwt/v4 v4.5.0 // indirect
github.com/golang-jwt/jwt/v5 v5.1.0 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.3 // indirect
github.com/google/gnostic-models v0.6.8 // indirect
github.com/google/go-querystring v1.1.0 // indirect
github.com/google/gofuzz v1.2.0 // indirect
github.com/google/s2a-go v0.1.7 // indirect
github.com/google/uuid v1.4.0 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.2.5 // indirect
Expand All @@ -98,19 +109,26 @@ require (
github.com/hashicorp/go-tfe v1.2.0 // indirect
github.com/hashicorp/hcl v1.0.0 // indirect
github.com/hashicorp/jsonapi v0.0.0-20210826224640-ee7dae0fb22d // indirect
github.com/imdario/mergo v0.3.12 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/itchyny/gojq v0.12.11 // indirect
github.com/itchyny/timefmt-go v0.1.5 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
github.com/josharian/intern v1.0.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect
github.com/kylelemons/godebug v1.1.0 // indirect
github.com/lib/pq v1.10.9 // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.17 // indirect
github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/mitchellh/go-wordwrap v1.0.1 // indirect
github.com/mitchellh/mapstructure v1.5.0 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/opentracing/opentracing-go v1.2.0 // indirect
github.com/pkg/browser v0.0.0-20210911075715-681adbf594b8 // indirect
github.com/pkg/errors v0.9.1 // indirect
Expand All @@ -120,7 +138,6 @@ require (
github.com/sirupsen/logrus v1.9.3 // indirect
github.com/spf13/cobra v1.7.0 // indirect
github.com/spf13/pflag v1.0.5 // indirect
github.com/tidwall/gjson v1.17.0 // indirect
github.com/tidwall/match v1.1.1 // indirect
github.com/tidwall/pretty v1.2.0 // indirect
github.com/uber/jaeger-client-go v2.30.0+incompatible // indirect
Expand All @@ -134,7 +151,7 @@ require (
golang.org/x/sys v0.14.0 // indirect
golang.org/x/term v0.14.0 // indirect
golang.org/x/text v0.14.0 // indirect
golang.org/x/time v0.0.0-20220609170525-579cf78fd858 // indirect
golang.org/x/time v0.3.0 // indirect
golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20230822172742-b8732ec3820d // indirect
Expand All @@ -143,5 +160,14 @@ require (
google.golang.org/grpc v1.58.3 // indirect
google.golang.org/protobuf v1.31.0 // indirect
gopkg.in/gookit/color.v1 v1.1.6 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
k8s.io/api v0.28.4 // indirect
k8s.io/klog/v2 v2.100.1 // indirect
k8s.io/kube-openapi v0.0.0-20230717233707-2695361300d9 // indirect
k8s.io/utils v0.0.0-20230406110748-d93618cff8a2 // indirect
sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect
sigs.k8s.io/structured-merge-diff/v4 v4.2.3 // indirect
sigs.k8s.io/yaml v1.3.0 // indirect
)
Loading