zstd: aws: compress: fix incompatible pointer glitches #10833

cosmo0920 · 2025-09-04T07:26:35Z

This is following up PR for #10794.
From gcc-14 or above, a check of incompatible pointer types was starting to be quite struct behavior.
And I found that public functions on flb_zstd.c are not needed to use size_t as return types because they just return their status codes.
So, we need to adjust their return types and to remove needless type castings.

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

Example configuration file for the change
Debug log output from testing the change

Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

Run local packaging test showing all targets (including any new ones) build.
Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

Documentation required for this feature

Backporting

Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

Summary by CodeRabbit

Refactor
- Zstandard compression/decompression helpers now return status codes instead of sizes, improving clarity and consistency. Integrations that rely on previous return values may need updates.
Bug Fixes
- Improved error signaling during compression/decompression to prevent misinterpreting failures as valid sizes.
Tests
- Updated test cases to align with the new status-based return behavior for compression/decompression helpers.

Signed-off-by: Hiroshi Hatake <[email protected]>

coderabbitai · 2025-09-04T07:26:42Z

Walkthrough

Return types of flb_zstd_compress and flb_zstd_uncompress changed from size_t to int across header and implementation. Callers in AWS compression code and tests were updated to use direct function pointers matching the new signatures. No other logic or control flow changes were introduced.

Changes

Cohort / File(s)	Summary
ZSTD API signature change `include/fluent-bit/flb_zstd.h`, `src/flb_zstd.c`	Change return type of `flb_zstd_compress` and `flb_zstd_uncompress` from `size_t` to `int` (0 on success, -1 on error); declarations and definitions synchronized.
AWS compression integration `src/aws/flb_aws_compress.c`	Update zstd entry to use direct function pointer `&flb_zstd_compress` without cast to match new `int` return type.
Tests update `tests/internal/aws_compress.c`	Pass `&flb_zstd_uncompress` directly to test helper, removing cast and aligning with `int(*)(...)` signature.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor Caller
  participant ZSTD_API as flb_zstd_* (compress/uncompress)
  participant ZSTD as libzstd

  Caller->>ZSTD_API: call(in_data, in_len, &out_data, &out_len)
  ZSTD_API->>ZSTD: perform (de)compression
  alt success
    ZSTD-->>ZSTD_API: result OK
    ZSTD_API-->>Caller: return 0, out_data/out_len set
  else error
    ZSTD-->>ZSTD_API: error
    ZSTD_API-->>Caller: return -1
  end

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

out_s3: Add ZSTD Compression Support #10794 — Adjusts flb_zstd API return types and updates callers; directly overlaps with the same function signature changes and usage patterns.

Suggested labels

docs-required, backport to v4.0.x

Suggested reviewers

edsiper
koleini
fujimotos
PettitWesley

Poem

A hop and a bop through headers I go,
Swapping size_t for int—status in tow.
Pointers now tidy, no casts in the breeze,
Compress, uncompress, with elegant ease.
Carrots compiled, tests nibble and grin—
ZSTD’s ready; thump-thump, merge it in! 🥕🐇

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

MCP integration is disabled by default for public repositories
Jira integration is disabled by default for public repositories
Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between f27ac1f and 2867312.

📒 Files selected for processing (4)

include/fluent-bit/flb_zstd.h (1 hunks)
src/aws/flb_aws_compress.c (1 hunks)
src/flb_zstd.c (2 hunks)
tests/internal/aws_compress.c (1 hunks)

🧰 Additional context used

🧠 Learnings (7)

📓 Common learnings

Learnt from: shadowshot-x
PR: fluent/fluent-bit#10794
File: tests/internal/aws_compress.c:39-42
Timestamp: 2025-08-29T06:24:26.170Z
Learning: In Fluent Bit, ZSTD compression support is enabled by default and does not require conditional compilation guards (like #ifdef FLB_HAVE_ZSTD) around ZSTD-related code declarations and implementations.

Learnt from: shadowshot-x
PR: fluent/fluent-bit#10794
File: tests/internal/aws_compress.c:7-7
Timestamp: 2025-08-29T06:25:02.561Z
Learning: In Fluent Bit, ZSTD (zstandard) compression library is bundled directly in the source tree at `lib/zstd-1.5.7` and is built unconditionally as a static library. Unlike optional external dependencies, ZSTD does not use conditional compilation guards like `FLB_HAVE_ZSTD` and is always available. Headers like `<fluent-bit/flb_zstd.h>` can be included directly without guards.

Learnt from: shadowshot-x
PR: fluent/fluent-bit#10794
File: tests/internal/aws_compress.c:93-107
Timestamp: 2025-08-29T06:25:27.250Z
Learning: In Fluent Bit, ZSTD compression is enabled by default and is treated as a core dependency, not requiring conditional compilation guards like `#ifdef FLB_HAVE_ZSTD`. Unlike some other optional components such as ARROW/PARQUET (which use `#ifdef FLB_HAVE_ARROW` guards), ZSTD support is always available and doesn't need build-time conditionals. ZSTD headers are included directly without guards across multiple plugins and core components.

Learnt from: shadowshot-x
PR: fluent/fluent-bit#10794
File: src/aws/flb_aws_compress.c:52-56
Timestamp: 2025-08-29T06:24:55.855Z
Learning: ZSTD compression is always available in Fluent Bit and does not require conditional compilation guards. Unlike Arrow/Parquet which use #ifdef FLB_HAVE_ARROW guards, ZSTD is built unconditionally with flb_zstd.c included directly in src/CMakeLists.txt and a bundled ZSTD library at lib/zstd-1.5.7/.

Learnt from: shadowshot-x
PR: fluent/fluent-bit#10794
File: tests/internal/aws_compress.c:93-107
Timestamp: 2025-08-29T06:25:27.250Z
Learning: In Fluent Bit, ZSTD compression is enabled by default and is treated as a core dependency, not requiring conditional compilation guards like `#ifdef FLB_HAVE_ZSTD`. Unlike some other optional components, ZSTD support is always available and doesn't need build-time conditionals.

📚 Learning: 2025-08-29T06:25:02.561Z

Learnt from: shadowshot-x
PR: fluent/fluent-bit#10794
File: tests/internal/aws_compress.c:7-7
Timestamp: 2025-08-29T06:25:02.561Z
Learning: In Fluent Bit, ZSTD (zstandard) compression library is bundled directly in the source tree at `lib/zstd-1.5.7` and is built unconditionally as a static library. Unlike optional external dependencies, ZSTD does not use conditional compilation guards like `FLB_HAVE_ZSTD` and is always available. Headers like `<fluent-bit/flb_zstd.h>` can be included directly without guards.

Applied to files:

src/aws/flb_aws_compress.c
include/fluent-bit/flb_zstd.h
src/flb_zstd.c

📚 Learning: 2025-08-29T06:24:26.170Z

Learnt from: shadowshot-x
PR: fluent/fluent-bit#10794
File: tests/internal/aws_compress.c:39-42
Timestamp: 2025-08-29T06:24:26.170Z
Learning: In Fluent Bit, ZSTD compression support is enabled by default and does not require conditional compilation guards (like #ifdef FLB_HAVE_ZSTD) around ZSTD-related code declarations and implementations.

Applied to files:

src/aws/flb_aws_compress.c
include/fluent-bit/flb_zstd.h
src/flb_zstd.c

📚 Learning: 2025-08-29T06:25:27.250Z

Learnt from: shadowshot-x
PR: fluent/fluent-bit#10794
File: tests/internal/aws_compress.c:93-107
Timestamp: 2025-08-29T06:25:27.250Z
Learning: In Fluent Bit, ZSTD compression is enabled by default and is treated as a core dependency, not requiring conditional compilation guards like `#ifdef FLB_HAVE_ZSTD`. Unlike some other optional components such as ARROW/PARQUET (which use `#ifdef FLB_HAVE_ARROW` guards), ZSTD support is always available and doesn't need build-time conditionals. ZSTD headers are included directly without guards across multiple plugins and core components.

Applied to files:

src/aws/flb_aws_compress.c
include/fluent-bit/flb_zstd.h
src/flb_zstd.c

📚 Learning: 2025-08-29T06:24:55.855Z

Learnt from: shadowshot-x
PR: fluent/fluent-bit#10794
File: src/aws/flb_aws_compress.c:52-56
Timestamp: 2025-08-29T06:24:55.855Z
Learning: ZSTD compression is always available in Fluent Bit and does not require conditional compilation guards. Unlike Arrow/Parquet which use #ifdef FLB_HAVE_ARROW guards, ZSTD is built unconditionally with flb_zstd.c included directly in src/CMakeLists.txt and a bundled ZSTD library at lib/zstd-1.5.7/.

Applied to files:

src/aws/flb_aws_compress.c
include/fluent-bit/flb_zstd.h
src/flb_zstd.c

📚 Learning: 2025-08-29T06:25:27.250Z

Learnt from: shadowshot-x
PR: fluent/fluent-bit#10794
File: tests/internal/aws_compress.c:93-107
Timestamp: 2025-08-29T06:25:27.250Z
Learning: In Fluent Bit, ZSTD compression is enabled by default and is treated as a core dependency, not requiring conditional compilation guards like `#ifdef FLB_HAVE_ZSTD`. Unlike some other optional components, ZSTD support is always available and doesn't need build-time conditionals.

Applied to files:

src/aws/flb_aws_compress.c
include/fluent-bit/flb_zstd.h
src/flb_zstd.c

📚 Learning: 2025-08-29T06:24:44.797Z

Learnt from: shadowshot-x
PR: fluent/fluent-bit#10794
File: src/aws/flb_aws_compress.c:26-26
Timestamp: 2025-08-29T06:24:44.797Z
Learning: In Fluent Bit, ZSTD support is always available and enabled by default. The build system automatically detects and uses either the system libzstd library or builds the bundled ZSTD version. Unlike other optional dependencies like Arrow which use conditional compilation guards (e.g., FLB_HAVE_ARROW), ZSTD does not require conditional includes or build flags.

Applied to files:

src/aws/flb_aws_compress.c
include/fluent-bit/flb_zstd.h

🧬 Code graph analysis (3)

tests/internal/aws_compress.c (1)

src/flb_zstd.c (1)

flb_zstd_uncompress (130-163)

src/aws/flb_aws_compress.c (1)

src/flb_zstd.c (1)

flb_zstd_compress (30-54)

include/fluent-bit/flb_zstd.h (1)

src/flb_zstd.c (2)

flb_zstd_compress (30-54)

flb_zstd_uncompress (130-163)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (26)

GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, gcc, g++, ubuntu-22.04, clang-12)
GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-24.04, clang-14)
GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-22.04, clang-12)
GitHub Check: PR - fuzzing test
GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, clang, clang++)
GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, gcc, g++, ubuntu-24.04, clang-14)
GitHub Check: pr-compile-centos-7
GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_ARROW=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_COVERAGE=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, clang, clang++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, gcc, g++)
GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, clang, clang++)

🔇 Additional comments (5)

include/fluent-bit/flb_zstd.h (1)
26-27: Insert ZSTD API docs and verify external dependents
Add above the prototypes in include/fluent-bit/flb_zstd.h:
 /* Returns:
  *   0 on success, -1 on error.
  * On success:
  *   - *out_data points to a malloc'd buffer that the caller must free with flb_free().
  *   - *out_len contains the number of bytes in *out_data.
  */
All internal callers capture the return in an int and check for 0; no size_t assignments or casts found.

Verify no out-of-tree consumers rely on the old size_t return value.
src/flb_zstd.c (1)

30-54: LGTM: status-code return for flb_zstd_compress matches callers and removes GCC 14 pointer cast issues

No functional concerns; allocation, error handling, and out parameters are correct.

src/aws/flb_aws_compress.c (2)

53-56: Direct function pointer (&flb_zstd_compress) fixes incompatible-pointer-type warnings

Matches the table’s expected signature and the new int return type. Good cleanup.

45-56: No stale casts or mismatched prototypes remain
Verified that flb_zstd_compress/flb_zstd_uncompress signatures match the compression_option pointer type (returning int), with no casts or size_t-return declarations, and all callers assign to int ret and check for 0/-1.

tests/internal/aws_compress.c (1)

272-278: Correct: pass proper decompress callback type for zstd

Using &flb_zstd_uncompress matches the updated int(*)(...)-return signature and removes the bad cast.

✨ Finishing Touches

📝 Generate Docstrings

🧪 Generate unit tests

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch cosmo0920-fix-incompatible-pointer-glitches

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>, please review it.
- Open a follow-up GitHub issue for this discussion.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

Add @coderabbitai ignore or @coderabbit ignore anywhere in the PR description to prevent this PR from being reviewed.
Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (`.coderabbit.yaml`)

You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
Please see the configuration documentation for more information.
If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Status, Documentation and Community

Visit our Status Page to check the current availability of CodeRabbit.
Visit our Documentation for detailed information on how to use CodeRabbit.
Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

edsiper · 2025-09-04T15:22:51Z

@cosmo0920 is there any other component that needs to be adjusted ? (I see others are using zstd)

cosmo0920 · 2025-09-05T02:52:27Z

@cosmo0920 is there any other component that needs to be adjusted ? (I see others are using zstd)

No, raw usages of size_t and int are interchangeable with lossy conversions. But, when making to use pointer, it's incompatible.

cosmo0920 added 3 commits September 4, 2025 16:22

zstd: Use int type for return type of public functions

48c0f23

Signed-off-by: Hiroshi Hatake <[email protected]>

aws: compress: Remove a needless type casting

ea3b9a6

Signed-off-by: Hiroshi Hatake <[email protected]>

tests: internal: aws_compress: Remove a needless type casting

2867312

Signed-off-by: Hiroshi Hatake <[email protected]>

cosmo0920 requested review from PettitWesley, edsiper, fujimotos and koleini as code owners September 4, 2025 07:26

github-actions bot added the docs-required label Sep 4, 2025

cosmo0920 had a problem deploying to pr September 4, 2025 07:26 — with GitHub Actions Failure

cosmo0920 added this to the Fluent Bit v4.1 milestone Sep 4, 2025

cosmo0920 temporarily deployed to pr September 4, 2025 07:30 — with GitHub Actions Inactive

cosmo0920 temporarily deployed to pr September 4, 2025 07:45 — with GitHub Actions Inactive

cosmo0920 temporarily deployed to pr September 4, 2025 07:46 — with GitHub Actions Inactive

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

zstd: aws: compress: fix incompatible pointer glitches #10833

zstd: aws: compress: fix incompatible pointer glitches #10833

cosmo0920 commented Sep 4, 2025 •

edited by coderabbitai bot

Loading

Uh oh!

coderabbitai bot commented Sep 4, 2025 •

edited

Loading

Chat

Support

CodeRabbit Commands (Invoked using PR/Issue comments)

Other keywords and placeholders

CodeRabbit Configuration File (`.coderabbit.yaml`)

Status, Documentation and Community

Uh oh!

edsiper commented Sep 4, 2025

Uh oh!

cosmo0920 commented Sep 5, 2025 •

edited

Loading

Uh oh!

Uh oh!

zstd: aws: compress: fix incompatible pointer glitches #10833

Are you sure you want to change the base?

zstd: aws: compress: fix incompatible pointer glitches #10833

Conversation

cosmo0920 commented Sep 4, 2025 • edited by coderabbitai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

coderabbitai bot commented Sep 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Sequence Diagram(s)

Estimated code review effort

Possibly related PRs

Suggested labels

Suggested reviewers

Poem

Chat

Support

CodeRabbit Commands (Invoked using PR/Issue comments)

Other keywords and placeholders

CodeRabbit Configuration File (.coderabbit.yaml)

Status, Documentation and Community

Uh oh!

edsiper commented Sep 4, 2025

Uh oh!

cosmo0920 commented Sep 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

cosmo0920 commented Sep 4, 2025 •

edited by coderabbitai bot

Loading

coderabbitai bot commented Sep 4, 2025 •

edited

Loading

CodeRabbit Configuration File (`.coderabbit.yaml`)

cosmo0920 commented Sep 5, 2025 •

edited

Loading