feat: Add Secret Filtering component #1593
Conversation
|
|
romain-gaillard
changed the title
romain-gaillard
requested review from
clayton-cornell and
a team
as code owners
| // | ||
| // For the first case, we can replace the entire match with the redaction string. | ||
| // For the second case, we can replace the first submatch with the redaction string (to avoid redacting something else than the secret such as delimiters). | ||
| for _, occ := range r.regex.FindAllStringSubmatch(entry.Line, -1) { |
There was a problem hiding this comment.
Think we can split this block into a separate func, lots of deep nesting going on.
| } | ||
|
|
||
| // Update implements component.Component. | ||
| func (c *Component) Update(args component.Arguments) error { |
There was a problem hiding this comment.
Update should likely include the work your doing in new, ie update should handle them changing the toml file or any other field. So maybe a shared func for recreating the internal representation.
There was a problem hiding this comment.
Changed in d8fa6e5.
Actually, because New(...) calls Update(...) anyway, I moved the code there so that it's run both when the component is created and updated 😄
clayton-cornell
added
the
type/docs
Thanks a lot! I addressed your suggestions in 500891f 😄 |
| } | ||
| // If allowed, skip redaction | ||
| if allowRule != nil { | ||
| level.Info(c.opts.Logger).Log("msg", "secret in allowlist", "rule", r.name, "source", allowRule.Source) |
There was a problem hiding this comment.
this should be on "Debug" level
|
|
||
| func init() { | ||
| component.Register(component.Registration{ | ||
| Name: "loki.secretfilter", |
There was a problem hiding this comment.
Could you add live debugging support? See this PR on how to add it: https://github.com/grafana/alloy/pull/1304/files
It would be nice for users to see the affected log lines before and after the process to make sure that the redaction is working as expected for them. Of course, if they expose the endpoint in prod they should not let the live debugging enabled but there are warnings in the doc about this
|
|
||
| {{< docs/shared lookup="stability/experimental.md" source="alloy" version="<ALLOY_VERSION>" >}} | ||
|
|
||
| `loki.secretfilter` receives log entries and redacts sensitive information, such as secrets, from them. |
There was a problem hiding this comment.
it would be nice to have a little bit more information on how the redaction is performed. It "only" redacts sensitive information that matches the provided regex (custom or the one in the embedded file). It's possible that sensitive information still passes through the net; users should be warned not to rely blindly on the component.
| for _, t := range c.args.Types { | ||
| if strings.HasPrefix(strings.ToLower(rule.ID), strings.ToLower(t)) { | ||
| found = true | ||
| continue |
There was a problem hiding this comment.
| continue | |
| break |
| secret = occ[r.secretGroup] | ||
| } else { | ||
| // If not and there are two submatches, the first one is the secret | ||
| if len(occ) == 2 { |
There was a problem hiding this comment.
should this just be an "else if" ?
PR Description
This PR adds a secret filtering component for Loki into Alloy. It is based on the work we have done in the August 2024 hackathon.
Which issue(s) this PR fixes
Notes to the Reviewer
PR Checklist