Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

out_rdkafka2: add patch for new version of rdkafka #505

Merged
merged 3 commits into from
Jul 31, 2024

Conversation

ttych
Copy link

@ttych ttych commented Dec 9, 2023

All PR are failing now because of rdkafka gem update.

The current code works with rdkafka 0.12.0,
and in 0.14.0 (there is no 0.13.0), the Rdkafka::Producer implementation has changed,
no more @client and Rdkafka::Producer::Client,
now there is a @native_client and Rdkafka::NativeKafka.

No update on this part of the code between 0.14.0 and 0.15.0.

There is a constraint in the out_rdkafka2, on the Rdkafka::Producer#close, to add a timeout,
which is not part of the standard gem.

I try to update this behavior since following the current way to patch.

Thanks.

README.md Show resolved Hide resolved
@ttych ttych force-pushed the rdkafka-update branch 2 times, most recently from e3fa121 to 12d1f2e Compare December 9, 2023 12:11
@ttych
Copy link
Author

ttych commented Dec 10, 2023

I just see #500,
so this is not a safe way to do it this way.

and also, waterdrop encapsulation for rdkafka is just for producer side.

Copy link

This PR has been automatically marked as stale because it has been open 90 days with no activity. Remove stale label or comment or this PR will be closed in 30 days

Copy link
Member

@ashie ashie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for my late response, I overlooked this.
It looks good, but need to update for rdkafka-ruby v0.16.

Hey @kenhys, I'm now considering to revert #513 then apply this with some further improvements. In addition, we may need to feedback this timeout patch to upstream to avoid future monkey patching. What do you think about it?

lib/fluent/plugin/rdkafka_patch/0_14_0.rb Show resolved Hide resolved
@ashie ashie added this to the v0.19.3 milestone Jul 30, 2024
@kenhys
Copy link
Contributor

kenhys commented Jul 31, 2024

I'm not sure yet but I agreed with reverting #513 to improve it.

@kenhys kenhys force-pushed the rdkafka-update branch 2 times, most recently from 09f3f76 to 8b78582 Compare July 31, 2024 03:35
@kenhys
Copy link
Contributor

kenhys commented Jul 31, 2024

In this approach, the border case should also follow in CI.

@kenhys
Copy link
Contributor

kenhys commented Jul 31, 2024

Reverted #513.

Copy link
Member

@ashie ashie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ashie ashie merged commit b6b50a0 into fluent:master Jul 31, 2024
19 checks passed
@ashie
Copy link
Member

ashie commented Jul 31, 2024

Thanks for your contribution! @ttych

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants