Get streaming hash aggregate back #913
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In multi-phase aggregate, this feature streams entries of the bottom phase when out of memory instead of spilling to disk, it avoids the disk I/O and roughly deduplicates to save the network I/O. The planner always supports streaming aggregate and there are lots of cases, but the executor part was lost while merging PostgreSQL 12, this commit gets it back.
|
I retain the original commit message from GPDB since most of the work was done by them, not me. |
|
Looks like CI need simple restart |
|
I guess this is related to #914 so posting to link them |
reshke
approved these changes
Feb 11, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR reintroduces streaming hash aggregation functionality to CBDB (Cloudberry Database). While it is inspired by the implementation in GPDB, it is not a straightforward cherry-pick due to significant differences between GPDB and CBDB, particularly because CBDB is based on the PostgreSQL 14 kernel. As a result, I had to make several adjustments to resolve conflicts and adapt the code to the CBDB codebase.
For example:
The
lookup_hash_entryfunction, which was modified in GPDB, has been removed in PostgreSQL 14. I adapted the code to work with the current CBDB implementation.Some fault injection mechanisms from GPDB could not be ported directly due to incompatibilities or missing dependencies in CBDB. For now, these fault injection points have been removed to ensure that streaming hash aggregation works correctly in CBDB. Future work will focus on reintroducing these mechanisms once the necessary infrastructure is in place.
Checklist
This PR lays the groundwork for streaming hash aggregation in CBDB and ensures compatibility with the PostgreSQL 14 kernel. Your feedback and review are greatly appreciated!
Let me know if you need further adjustments!
Fixes #ISSUE_Number
What does this PR do?
Type of Change
Breaking Changes
Test Plan
make installcheckmake -C src/test installcheck-cbdb-parallelImpact
Performance:
User-facing changes:
Dependencies:
Checklist
Additional Context
CI Skip Instructions