enhancement(kubernetes_logs source): Wire up acknowledgement support via existing file ack support

[ganelo]

Summary

Mirror changes made to file source as part of implementing acknowledgements support there over to kubernetes_logs source.

Change Type

• Bug fix
• New feature
• Non-functional (chore, refactoring, docs)
• Performance

Is this a breaking change?

• Yes
• No

How did you test this PR?

Given that the underlying feature work for implementing acknowledgements in the file source is well-tested, my plan is to deploy a custom build of vector with this change to a dev environment and simply smoke test to confirm that, when acknowledgements are enabled via config, we see evidence that log events are not considered processed until they are fully delivered at the end of the sink chain (eg by counting unique log event ids at the end of the test after bouncing some of the intermediate hops in chain during the test).

Does this PR include user facing changes?

• Yes. Please add a changelog fragment based on our guidelines.
• No. A maintainer will apply the "no-changelog" label to this PR.

Checklist

• Please read our Vector contributor resources.
  • make check-all is a good command to run locally. This check is
    defined here. Some of these
    checks might not be relevant to your PR. For Rust changes, at the very least you should run:
    • cargo fmt --all
    • cargo clippy --workspace --all-targets -- -D warnings
    • cargo nextest run --workspace (alternatively, you can run cargo test --all)
• If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
  run dd-rust-license-tool write to regenerate the license inventory and commit the changes (if any). More details here.

References

• Closes: Add support for end-to-end acknowledgements to kubernetes_logs source #7462

• Ref Add support for end-to-end acknowledgements to all relevant sources #7336 and Add support for end-to-end acknowledgements to all relevant sources #9856 for global tracking of ACK support across sources
• Ref Add support for end-to-end acknowledgements to file source #7459 for ACK support in file source, which this PR depends on (implemented in enhancement(file source): Add support for acknowledgements #7816)
• Ref Add support for end-to-end acknowledgements to all relevant sources #9856 (which was closed as a dupe of 7462)

[pront]

Hi @ganelo and thank you for this PR. The make check-component-docs check failed. Can you take a look?

[ganelo]

Hi @ganelo and thank you for this PR. The make check-component-docs check failed. Can you take a look?

Hi @pront - this seems to be broken on master, as far as I can tell. In any case, nothing I've touched should have broken the ruby script. Not sure how best to proceed.

[jszwedko]

Hi @ganelo and thank you for this PR. The make check-component-docs check failed. Can you take a look?

Hi @pront - this seems to be broken on master, as far as I can tell. In any case, nothing I've touched should have broken the ruby script. Not sure how best to proceed.

Can you share the failure? The script requires Ruby 3, which often trips people up.

[pront]

Hi @pront - this seems to be broken on master, as far as I can tell.

I verified that it is not broken master.

In any case, nothing I've touched should have broken the ruby script. Not sure how best to proceed.

This PR adds configuration fields and we rely on source code to generate docs. Hence, docs need to regenerated. As @jszwedko pointed out we have a dependency on Ruby 3. My version locally:

❯ ruby -v
ruby 3.1.4p223 (2023-03-30 revision 957bb7cb81) [arm64-darwin22]

[ganelo]

Ah okay, thanks for clarifying! Will try again tomorrow with the correct version of Ruby to get to the bottom of the failure.

…

On Thu, Feb 6, 2025, 18:37 Pavlos Rontidis ***@***.***> wrote: Hi @pront <https://github.com/pront> - this seems to be broken on master, as far as I can tell. I verified that it is not broken master. In any case, nothing I've touched should have broken the ruby script. Not sure how best to proceed. This PR adds configuration fields and we rely on source code to generate docs. Hence, docs need to regenerated. As @jszwedko <https://github.com/jszwedko> pointed out we have a dependency on Ruby 3. My version locally: ❯ ruby -v ruby 3.1.4p223 (2023-03-30 revision 957bb7cb81) [arm64-darwin22] — Reply to this email directly, view it on GitHub <#22366 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB4OKWXU3ELUEEVUJQVLFDD2OPW3BAVCNFSM6AAAAABWPOJTRWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNBRGM2TKNRZGY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[pront]

Looks better now 👍 Is it possible to add a test to cover the acks feature? You can draw inspiration from other sources e.g. the file source:

vector/src/sources/file.rs

Lines 2375 to 2376 in b90715b

	let (tx, rx) = if acking_mode == Acks {
	let (tx, rx) = SourceSender::new_test_finalize(EventStatus::Delivered);

[ganelo]

Looks better now 👍 Is it possible to add a test to cover the acks feature? You can draw inspiration from other sources e.g. the file source:

vector/src/sources/file.rs

Lines 2375 to 2376 in b90715b

	let (tx, rx) = if acking_mode == Acks {
	let (tx, rx) = SourceSender::new_test_finalize(EventStatus::Delivered);

To confirm - there aren't any existing tests that actually exercise the kubernetes_logs source, right? I poked around and didn't find much that did anything beyond some niche unit tests. Certainly nothing to mirror the run_file_source fn of file.rs.

I'm not utterly opposed to writing one from more or less whole cloth, but it will be somewhat slow going as I'm a relative newcomer to Rust.