Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When will it support amqp ? we got performance issue when use https. #13

Open
gaussye opened this issue Nov 22, 2018 · 2 comments
Open

Comments

@gaussye
Copy link

gaussye commented Nov 22, 2018

We collected large of logs through fluentd azure eventhub plugins, however the problem is that this plugin use https instead of amqp. Any plan to implement amqp?

@sputnik13
Copy link

We were also having performance issues until we switched to persistent connection and tweaked the output buffer settings, below is the output buffer setting we're using

<match **>
   @type azureeventhubs_buffered
   @id out_aeh_docker
   connection_string "#{ENV['FLUENT_EVENTHUB_CONNECTION_STRING']}"
   hub_name "#{ENV['FLUENT_EVENTHUB_HUB_NAME']}"
   include_tag true
   type https
   message_properties {"Type": "fluentd"}
   <buffer>
      total_limit_size "2GB"
      chunk_full_threshold "0.8"
      compress gzip
      flush_interval "5s"
      flush_thread_count "10"
   </buffer>
</match>

If this doesn't help you might also want to try https://github.com/sputnik13/fluent-plugin-azureeventhubs a try? You'd have to put it in your Gemfile as

gem "fluent-plugin-azureeventhubs", :git => 'https://github.com/sputnik13/fluent-plugin-azureeventhubs', :ref => 'persistent_connection'

@danmassie
Copy link

Using the persistent connection from @sputnik13 fork fixed it for me too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants