I think we try to avoid adding the -devel ones as they usually introduce a load of extra dependencies so is that required?

Do you mean these need to be added to the builder image as well? None are present on either it seems so we must add to the builder at least but then what needs to be in the production images?

https://github.com/fluent/fluent-bit/blob/master/dockerfiles/Dockerfile

What features to they enable?

ah yea I think we can avoid devel and only keep the cyrus-sasl-gssapi as the new one. In order to build it I would assume we need it on the builder too

This would mainly be for SASL features within rdkafka plugin for Fluent Bit

Yeah typically you need the devel ones in the builder stage but then only the runtime ones for the production images. I think a lot of this is auto configured during build as well so it may be we only need to add the dependencies to the relevant stages.

I'll pick this up shortly

[SERVICE]
    log_level debug
[INPUT]
    Name kafka
    brokers localhost:4511
    topics MSKTutorialTopic
    rdkafka.security.protocol sasl_ssl
    rdkafka.sasl.mechanism OAUTHBEARER
    rdkafka.sasl.oauthbearer.method oidc
    rdkafka.sasl.oauthbearer.client.id xxxx
    rdkafka.sasl.oauthbearer.client.secret xxxx
    rdkafka.sasl.oauthbearer.token.endpoint.url http://localhost:4566/_aws/cognito-idp/oauth2/token
    rdkafka.sasl.oauthbearer.scope kafka-cluster/read
    rdkafka.ssl.ca.location ./certs/ca.pem
    rdkafka.log_level 7
    rdkafka.log.thread.name true
    poll_ms 100


[OUTPUT]
    Name stdout
    match *

Compiled with

cmake -DFLB_KAFKA=On -DFLB_OUT_KAFKA=On -DWITH_CURL=ON -DWITH_SSL=ON -DWITH_SASL=ON ..