Skip to content

Optimize merging of partial case expression results #18152

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 46 commits into from
Oct 28, 2025
Merged

Optimize merging of partial case expression results #18152

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
e8698f7
Reduce record batch filtering in case_when_no_expr
pepijnve Oct 18, 2025
115b554
Reduce record batch filtering in case_when_with_expr
pepijnve Oct 18, 2025
9c641aa
Avoid unnecessary filtering in last iteration when no else expression…
pepijnve Oct 19, 2025
7539068
Use interleave to construct case result
pepijnve Oct 20, 2025
b4c1117
Add comments
pepijnve Oct 20, 2025
7eae3e4
Handle null where values correctly in `case_when_with_expr`
pepijnve Oct 21, 2025
f5448c9
Align `case_when_with_expr` and `case_when_no_expr`
pepijnve Oct 21, 2025
132757e
Exit early when all base values are null
pepijnve Oct 21, 2025
43c2fe2
Avoid calling `interleave` in simple cases
pepijnve Oct 21, 2025
f49d3ea
Formatting
pepijnve Oct 21, 2025
7e21a19
Merge branch 'main' into case_reduce_filtering
pepijnve Oct 21, 2025
9cb6496
Use a custom merge strategy that takes the case evaluation logic into…
pepijnve Oct 23, 2025
37f1334
Always optimize filters
pepijnve Oct 23, 2025
1b2942b
Remove accidental addition of pub
pepijnve Oct 23, 2025
f6a5734
Add comments regarding `RecordBatch::new_unchecked` usage
pepijnve Oct 23, 2025
3030557
Attempt to clarify merge logic
pepijnve Oct 23, 2025
cad318e
Rename arguments
pepijnve Oct 23, 2025
d57e7b6
Formatting
pepijnve Oct 23, 2025
7f60d68
More diagrams
pepijnve Oct 23, 2025
9cf46b2
More comments
pepijnve Oct 23, 2025
fc04bd5
Remove redundant comment
pepijnve Oct 23, 2025
7dba554
Calculate slice length once
pepijnve Oct 24, 2025
44b763c
Avoid filtering when all base values are null
pepijnve Oct 24, 2025
be578d4
Avoid filtering when branch matches all remaining rows
pepijnve Oct 24, 2025
53cb37b
Avoid expanding scalars in case_when_with_expr
pepijnve Oct 24, 2025
9ebe463
Update code comment
pepijnve Oct 25, 2025
2a6d1ae
Processing review comments
pepijnve Oct 25, 2025
58cdfa8
Remove links to private fields
pepijnve Oct 25, 2025
e385f08
Add debug assertion to detect duplicate row values.
pepijnve Oct 25, 2025
d512b2f
Add SLT with unreachable 1/0 in when predicate
pepijnve Oct 26, 2025
865bace
Add SLT that verifies that when branches are not evaluated for alread…
pepijnve Oct 26, 2025
17b6f9a
Introduce PartialResultIndex to reduce size of indices array and avoi…
pepijnve Oct 26, 2025
ba584d3
Formatting
pepijnve Oct 26, 2025
07119f8
Minor code cleanup
pepijnve Oct 26, 2025
160d425
Merge remote-tracking branch 'upstream/HEAD' into case_reduce_filtering
pepijnve Oct 27, 2025
3b8b6b8
Reorganise base_values null handling to avoid unnecessary computations
pepijnve Oct 27, 2025
d189c3b
Doc revisions
pepijnve Oct 27, 2025
596bd11
Assert row indices does not contain nulls
pepijnve Oct 27, 2025
8b079cc
Complete/single/covering naming consistency
pepijnve Oct 27, 2025
1afa6f6
Formatting
pepijnve Oct 27, 2025
878d86a
Add debug_assertions bounds check in merge
pepijnve Oct 27, 2025
2b5a168
Clarify usage of prep_null_mask_filter
pepijnve Oct 27, 2025
d08a916
Move debug assertion inside loop to remove loop duplication
pepijnve Oct 27, 2025
6fce213
Align with and without expr code
pepijnve Oct 27, 2025
dafea94
Clarify add_branch_result diagram
pepijnve Oct 27, 2025
7a4a24b
Add ResultState::Empty to defer creation of the indices array
pepijnve Oct 28, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading