Message filters: on deployment, job or origin #273
Conversation
|
I think I will change the "filter language" to something like and implement the available matchers similar as the But still, any feedback, esp. if this generally has a chance to be merged, would be great! |
hoegaarden
force-pushed
the
messagefilter
branch
from
0d9676c
to
fc74374
Compare
|
PR & PR description updated to the change of the "filter language". Some internal refactoring should make it easier to add new filters / getters; implementation is now saner (I think 🤷🏽). |
|
We also ran into this situation where we only want certain deployments to be exposed via firehose nozzle so a change like this would help us a lot in not flodding splunk with informations not needed... |
|
@rockb1017 @luckyj5 Can you please let me know if you'd consider merging that? Or point me to some other devs I could work with? |
|
@hoegaarden Thanks for reaching out. I have added this item to our backlog and it will be considered for the future release of the nozzle (not scheduled yet). Will reach out if any additional information is needed. Thanks! |
|
Also, can you please rebase this with |
hoegaarden
force-pushed
the
messagefilter
branch
from
fc74374
to
57779ec
Compare
|
@luckyj5 I have rebased my changes onto |
|
@hoegaarden, Yes, please raise it against develop branch. Thank you! |
@luckyj5 Done. Let me know what you think about those changes! |
This allows filtering of events based on the above mentioned fields.
Multiple filters can be configured and they will run against a message
in the order specified.
A filter has the form
field₀:comperator₀:value₀;field₁:comperator₁:value₁;…;fieldₙ,comperatorₙ,valueₙ
`field` is the messages field to check against.
`comperator` is how to compate the `field`'s value to the user provided value ('mustContain', 'mustNotContain')
`value` is the value the user want's to check messages for
Benchmarks have been added to get an understanding of the performance
impact of those filters.
hoegaarden
force-pushed
the
messagefilter
branch
from
57779ec
to
7f75adf
Compare
|
@harshit-splunk @kashyap-splunk @luckyj5 -- I saw that development has kicked in again. Please let me know if you'd consider merging my PR. Thanks. |
|
Sure, we will look into this as soon as we can and let you know. |
Similar to #175 & #162 we'd like to filter some messages before we push them out to splunk.
We have a use-case where we only want to forward stuff from any
p-healthwatch2-UUIDdeployment but not fromp-heathwatch2-expoerter-UUID. With this "filter language" this is possible with something likeThis allows us to deploy the same config on multiple foundations (with different deployment UUID for the same tile).
While for now we'd only need the filter on
deployment, I took the opportunity to also implement it forjobandorigin, as these can be implemented in the very same way. Additional filters could be implemented by implementing moreeventfilter.supportedGetters. Similar goes for the comparison functions:mustContainandmustNotContainare implemented, additional ones could be added by appending them toeventfilter.supportedFilters.Benchmarks have been implemented, those can be seen by running
make testvorginkgo ./eventrouter/.... According to those benchmarks, discarding messages is super fast & a single filter does not seem to have much of a performance impact. Multiple filters, though, might get a bit heavy. (BTW: happy to implement a limit on the numbers of filters allowed).If this get's accepted, I would also go forward and implement this in the tile.
cc: @rockb1017 @luckyj5 (because you cut the last releases)