Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

opt: add rule to decorrelate union operators #131141

Merged
merged 3 commits into from
Oct 3, 2024
Merged

opt: add rule to decorrelate union operators #131141

merged 3 commits into from
Oct 3, 2024

Commits on Oct 2, 2024

  1. opt: improve EnsureKey custom function 

    This commit makes a small improvement to the `EnsureKey` custom function
(used in decorrelation rules), so that it can add passthrough columns to
a `Project` operator in an effort to find a key. This can prevent rules
from adding unnecessary `Ordinality` operators to the query plan.

Epic: None

Release note: None
    @DrewKimball
    DrewKimball committed Oct 2, 2024
    Configuration menu
    Copy the full SHA
    5b364a5 View commit details
    Browse the repository at this point in the history

  2. opt: improve exists subquery hoisting 

    This commit makes a small improvement to the subquery-hoisting rules so
that hoisting an `EXISTS` subquery can often avoid projecting a new
column to check for NULL values. This can allow other optimization rules
to match later on.

Epic: None

Release note: None
    @DrewKimball
    DrewKimball committed Oct 2, 2024
    Configuration menu
    Copy the full SHA
    5bc3c1e View commit details
    Browse the repository at this point in the history

  3. opt: add rule to decorrelate unions in EXISTS subqueries 

    This commit adds a new rule `TryDecorrelateUnion`, which matches on a
`Union` or `UnionAll` operator in the input of a `ScalarGroupBy`. The
`ScalarGroupBy` must have "any-not-null" semantics, meaning it produces
an arbitrary non-null value from each input column.

If these conditions are satisfied, the `Union` operator is replaced by an
`InnerJoin` between two `ScalarGroupBy` operators. A `Project` coalesces
columns from each side of the join to produce the final aggregated values.

This transformation does not itself decorrelate the `Union` operators, but
it does make it easier for other rules to do so.

Release note: None

Epic: None
    @DrewKimball
    DrewKimball committed Oct 2, 2024
    Configuration menu
    Copy the full SHA
    1ec3568 View commit details
    Browse the repository at this point in the history