Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve memory efficiency in FnApiDoFnRunner #33522

Merged
merged 3 commits into from
Jan 14, 2025

Conversation

stankiewicz
Copy link
Contributor

Reduces memory footprint of storing map of coders, pcollections and windowStrategies within FnApiDoFnRunner by reusing the immutable component created once per stage vs copied recursively for earch transform.

@github-actions github-actions bot added the java label Jan 7, 2025
Copy link
Contributor

github-actions bot commented Jan 7, 2025

Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment assign set of reviewers

Copy link
Contributor

github-actions bot commented Jan 7, 2025

Assigning reviewers. If you would like to opt out of this review, comment assign to next reviewer:

R: @m-trieu for label java.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

@damccorm
Copy link
Contributor

@m-trieu would you mind taking a look at this one?

Copy link
Member

@kennknowles kennknowles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me.

@kennknowles
Copy link
Member

It is a small change that should impact all pipelines, so let's watch the postcommit test status. It seems pretty safe though so I think it is an adequate level of care to merge and watch how things go.

@kennknowles kennknowles merged commit 93476eb into apache:master Jan 14, 2025
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants