kafka source doesn't support non-PEM certificates

[tessneau]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

Closely tied to this issue: #22046

The doc for tls.crt_file for Kafka components states:

The certificate must be in DER, PEM (X.509), or PKCS#12 format. Additionally, the certificate can be provided as an inline string in PEM format.

However passing in a non-PEM certificate for example with this config

sources:
  in:
    type: kafka
    bootstrap_servers: localhost:9092
    group_id: test-group
    topics:
      - test-topic
    librdkafka_options:
      client.id: tess
      allow.auto.create.topics: "true"
      auto.offset.reset: latest
    tls:
      enabled: true
      crt_file: sample_crt_with_key.pfx

results in this error from librdkafka

Could not create Kafka consumer: Client creation error: ssl.certificate.location failed: error:0A080009:SSL routines::PEM lib

We should make it support non-PEM certificates that librdkafka supports.

Right now we set ssl.certificate.location here but we should probably be setting ssl.keystore.location if we recognize the file as PKCS#12

Configuration

No response

Version

vector 0.43.1 (aarch64-apple-darwin e30bf1f 2024-12-10 16:14:47.175528383)

Debug Output

No response

Example Data

No response

Additional Context

No response

References

• The kafka source and sink incorrectly document support for non-PEM certificates #22046