workflows: add sanity check for toolchains w/ CXX support

[ThomasDevoogdt]

Fluent-bit is written in C. There is no reason to have a CXX compiler. (Except for some libs which are all individually optional.) So add a check that enforces a C-only toolchain. E.g. useful for buildroot which does support platforms with only a C compiler.

Upstream status:

• lib: librdkafka: cmake: allow compilation without CXX support
  CMakeLists.txt: allow compilation without CXX support confluentinc/librdkafka#4366
• lib: zstd: cmake: only require a CXX compiler when tests are build
  [cmake] only require a CXX compiler when tests are build facebook/zstd#4357
• lib: luajit: cmake: do not require a CXX compiler
  N/A fluent-bit only
• lib: miniz: only require a C compiler
  CMakeLists.txt: only require a C compiler richgel999/miniz#338

[cosmo0920]

Hi, we don't want to modify the vendored libraries by ourselves.
Instead, could you send you patch into librdkafka repo?
https://github.com/confluentinc/librdkafka

[ThomasDevoogdt]

Hi, we don't want to modify the vendored libraries by ourselves. Instead, could you send you patch into librdkafka repo? https://github.com/confluentinc/librdkafka

Understand that, see my other PR/ remark #7765 (comment). There is an upstream PR: confluentinc/librdkafka#4366, but no reply in over a year.

[ThomasDevoogdt]

Hi @cosmo0920, is it possible to re-evaluate this? I don't get an answer from upstream, but the changes are relevant IMHO. So I think that we should go forward with it. I know that changes exists that this issue wrt librdkafka gets re-introduced when syncing upstream, but the added workflows will tell us about this at that time.

[ThomasDevoogdt]

@edsiper It has been a while. Any chance that this gets approved? I try to reduce my own patch list.

[ThomasDevoogdt]

@patrick-stephens @edsiper I see that 4.0.0 has landed, this is the ideal moment to consider this PR.

[patrick-stephens]

I think we have to merge the changes into the upstream dependencies rather than make them here as it is not sustainable to keep patching them each time we update. Running a full build again of another target seems a little time consuming, can we push that into one of the existing unit tests or similar builds instead we're doing anyway?

[ThomasDevoogdt]

I think we have to merge the changes into the upstream dependencies rather than make them here as it is not sustainable to keep patching them each time we update. Running a full build again of another target seems a little time consuming, can we push that into one of the existing unit tests or similar builds instead we're doing anyway?

Hi, I'm trying to upstream most of it. I will come back to this PR once done.

• CMakeLists.txt: only require a C compiler richgel999/miniz#338
• [cmake] only require a CXX compiler when tests are build facebook/zstd#4357
• CMakeLists.txt: allow compilation without CXX support confluentinc/librdkafka#4366
• build: only compile simdutf if there is CXX support #10277
• And for luajit, there is already a fluent-bit specific cmake file.

About having a separate build. I don't want to change existing builds, which are also compiling CXX features like simdutf.

[ThomasDevoogdt]

@cosmo0920 @edsiper There is only one commit pending for upstream. So I don't really see the big deal to not consider this PR. Any possible regression will be catched by the workflow file. Also, librdkafka is not a fast moving library, so I don't think it will break anytime soon.

[leonardo-albertovich]

Please remove the modifications to the dependencies, this PR should only change the workflow file in order to be mergeable.

[ThomasDevoogdt]

@leonardo-albertovich the workflow file won't pass the GH actions if the changes to the dependencies (I guess you mean the bundled libs) are not merged first. So not sure what exactly I have to do here. I guess it's not really allowed to add a workflow file which doesn't pass the tests.

[leonardo-albertovich]

Here's what I think we should do :

1. Remove the changes to the dependencies from your PR
2. Ensure that the dependency PRs are upstreamed
3. Update the dependencies
4. Rebase your PR on top of master
5. Merge your PR

[ThomasDevoogdt]

@leonardo-albertovich Ok, thanks for the clear explanation.

3. Update the dependencies

I have a PR open for all the dependencies, can you help to merge them?:

• lib: luajit: only require a C compiler #10318
• lib: zstd: only require a C compiler #10316
• lib: miniz: only require a C compiler #10315
• lib: librdkafka: only require a C compiler #10341

I will then rebase this branch.

[leonardo-albertovich]

I don't know if @edsiper prefers to make the updates himself once the upstream PRs are merged or if he's ok with those PRs you've opened.

I take that the PRs you're opening when the upstream PRs are merged are complete updates and not just what you changed right? Otherwise they obviously wouldn't be viable.

[ThomasDevoogdt]

I don't know if @edsiper prefers to make the updates himself once the upstream PRs are merged or if he's ok with those PRs you've opened.

Then I can wait for quite some time. Is it also requested to wait for the next upstream release tag?

I take that the PRs you're opening when the upstream PRs are merged are complete updates and not just what you changed right? Otherwise they obviously wouldn't be viable.

The changes I propose are quite minimal, I don't see why it can't be taken along as is. Otherwise, I don't see this branch merged in the first year. Also, the workflow file will anyway assert if things get out of sync.

[leonardo-albertovich]

That's up to @edsiper but please be mindful of the fact that even though fluent-bit might be a minor concern for you and you'll forget about it once you get your changes merged we will still be accountable for it so we want to ensure that things are done in a way that won't come back to bite us in the back in the future.

TBH I could've seen these dependency patches merged much faster if you implemented them as patches that are applied during the build process but you didn't seem to care so there's not much I can do to help you.

[ThomasDevoogdt]

TBH I could've seen these dependency patches merged much faster if you implemented them as patches that are applied during the build process but you didn't seem to care so there's not much I can do to help you.

@leonardo-albertovich, is there such system in fluent-bit to apply patches in the build process? If yes, then I can for sure use that. Or do you expect that I create such patch system? It obviously would make sense to have that.

And about caring, I do care about compatibility, before I started the Buildroot integration, a dozen of compilation targets were failing due to all kind of reasons. E.g. Libc usage (uClibc, musl, glibc), Linux versions, Builder back-ends (make, ninja), toolchains (GCC versions, with/without C++ support), and so on. I'm thankful that you guys accepted most of my PRs, this PR and a few others are some left overs.

FYI, this the Buildroot package: https://github.com/buildroot/buildroot/tree/master/package/fluent-bit.

[leonardo-albertovich]

There is no such system in fluent-bit and I just had to tell someone else about it in issue #10139 because we might need to patch the CMakeLists.txt file in zstd due to a compatibility issue for which they've had a PR open for about 7 years already so if you want to give it a try let me know, otherwise if the user from that issue confirms that the proposed change fixes the problem I might have to do it myself.

I think I'd do it within cmake and try to figure out something that doesn't require third party tools or something minimal because that could be a problem considering the platforms we support.