Added support for AWS MSK IAM

[mbartusiak]

AWS announced support for a new SASL mechanism called AWS_MSK_IAM which enables to handle both authentication and authorisation with AWS IAM. I added the dependency on aws-msk-iam-auth to enable support for that mechanism.

More on that:
https://aws.amazon.com/blogs/big-data/securing-apache-kafka-is-easy-and-familiar-with-iam-access-control-for-amazon-msk/

[fabioformosa]

Hi there!
Any news about this very helpful PR?
We really need this feature to use Kafdrop in our managed kafka in AWS with IAM authentication.

[manee2k6]

@mbartusiak Hi,
Did you added the client properties to support IAM access control.
ssl.truststore.location=<PATH_TO_TRUST_STORE_FILE>
security.protocol=SASL_SSL
sasl.mechanism=AWS_MSK_IAM
sasl.jaas.config=software.amazon.msk.auth.iam.IAMLoginModule required;
sasl.client.callback.handler.class=software.amazon.msk.auth.iam.IAMClientCallbackHandler

[mbartusiak]

@manee2k6
Yes, I passed those client properties and tested successfully that it works for an IAM enabled MSK cluster.

[manee2k6]

@mbartusiak It would be great if you can help me with the steps, i am trying to make changes in code, which is causing issue.
Your help here is highly appreciated.
email: [email protected] or zoom would do.

[fabioformosa]

I've followed these steps using the sourcecode of this PR:

• encoded in base64 this string

security.protocol=SASL_SSL
sasl.mechanism=AWS_MSK_IAM
sasl.jaas.config=software.amazon.msk.auth.iam.IAMLoginModule required;
sasl.client.callback.handler.class=software.amazon.msk.auth.iam.IAMClientCallbackHandler

• I've set to the kafdrop container the env:
  KAFKA_PROPERTIES=THE_ENCODED_STRING_AT_THE_PREV_STEP
• I get the error

ERROR [| kafdrop-admin] o.a.k.c.NetworkClient  : [AdminClient clientId=kafdrop-admin] Connection to node -2 (b-2.xxx.kafka.eu-south-1.amazonaws.com:9098) failed authentication due to: Access denied
WARN [| kafdrop-admin] o.a.k.c.a.i.AdminMetadataManager  : [AdminClient clientId=kafdrop-admin] Metadata update failed due to authentication error

I didn't specify the truststore because I read here https://aws.amazon.com/blogs/big-data/securing-apache-kafka-is-easy-and-familiar-with-iam-access-control-for-amazon-msk/

The ssl.truststore.location is unnecessary if your Java JDK distribution already trusts the TLS certificate of the MSK broker, which should be the case of all standard JDK distributions, because the TLS certificate is public.

Any ideas?

Must these props set?

--kafka.isSecured=true 
--kafka.env=some_env

Even if set, same error

[laxgoalie392]

any update on this? would be nice to see

[sali-cs]

Any update on this?

[rohan-changejar]

any progress on this auth mechanism, this capability would be highly appreciated

[Shreyank031]

Any updates... Waiting eagerly for this feature.

[Bert-R]

If you contribute a solution, we will review and merge it in the coming months. At the moment, we are even too busy to do such a review.