Missing customfields/extra_fields on Elasticsearch export

[qsoul]

Describe the bug

When using falco's append_output.extra_fields or falcosidekick customfields they both appear in falco's output, but are missing when exporting to Elasticsearch (via Elasticsearch output method). Those extra fields just are not present at all in the post request json body.

• Falco version:
  Falco version: 0.39.1

• Falcosidekick version:
  Falco version: 2.29.0

• Installation method:
  Kubernetes (via Helm)

[Issif]

Hi,

I think I was able to replicate the issue. Can you confirm me that the value elasticsearch.flattenfields is true in your config?

Edit: there's a bug in my code to "flatten" the keys of the output_fields, a check is missing, and the keys without any dot are removed. I'm on the fix right now

[Issif]

The PR with the fix is https://github.com/falcosecurity/falcosidekick/pull/1034/files, it will be in the next 2.30, sadly I don't have any ETA, but I could generate a release candidate for you if you need.

[qsoul]

Hi,

I think I was able to replicate the issue. Can you confirm me that the value elasticsearch.flattenfields is true in your config?

Edit: there's a bug in my code to "flatten" the keys of the output_fields, a check is missing, and the keys without any dot are removed. I'm on the fix right now

Thank you for handling it so fast!
I'll use the lastest image untill the next release. One more thanks!
p.s. I don't use elasticsearch.flattenfields explicitly, but use createindextemplate: true, so yes, that should be same I believe.

[Issif]

Yes, the createtemplate setting implies it, to avoid conflicts with some fields types that may appear as nested or not, we got multiple issues because of that