Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize BETWEEN predicate to include both the sides. #22521

Merged
merged 3 commits into from
Jul 23, 2024

Conversation

osscm
Copy link
Contributor

@osscm osscm commented Jun 27, 2024

Description

reviving this old PR

visitBetweenPredicate in SortExpressionVisitor can only handle one side of BETWEEN (GREATER_THAN_OR_EQUAL or LESS_THAN_OR_EQUAL), this pr optimize it and let it handle both sides of BETWEEN as conjunct expressions
Fixes one par of this issue: #22520

Additional context and related issues

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(x) Release notes are required, with the following suggested text:

# General
* Improve performance of inequality joins ({issue}`issuenumber`)

@cla-bot cla-bot bot added the cla-signed label Jun 27, 2024
@wendigo
Copy link
Contributor

wendigo commented Jun 27, 2024

Can you fix the test @osscm ?

@osscm osscm requested a review from sopel39 July 1, 2024 15:38
@@ -88,7 +88,7 @@ private static SortExpressionContext merge(SortExpressionContext left, SortExpre
}

private static class SortExpressionVisitor
extends IrVisitor<Optional<SortExpressionContext>, Void>
extends IrVisitor<Optional<List<SortExpressionContext>>, Void>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional is no longer needed, just List<SortExpressionContext>

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

@@ -115,7 +115,7 @@ protected Optional<SortExpressionContext> visitComparison(Comparison comparison,
hasBuildReferencesOnOtherSide = hasBuildSymbolReference(buildSymbols, comparison.right());
}
if (sortChannel.isPresent() && !hasBuildReferencesOnOtherSide) {
yield sortChannel.map(symbolReference -> new SortExpressionContext(symbolReference, singletonList(comparison)));
yield sortChannel.map(symbolReference -> singletonList(new SortExpressionContext(symbolReference, singletonList(comparison))));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: ImmutableList.of(...)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

@wendigo
Copy link
Contributor

wendigo commented Jul 22, 2024

@osscm please squash these two commits together - they are a single logical change

@osscm
Copy link
Contributor Author

osscm commented Jul 22, 2024

@osscm please squash these two commits together - they are a single logical change

TestCassandraConnectorTest>BaseConnectorTest.testCreateTableWithLongColumnName:3688

TestIcebergMinioOrcConnectorTest>BaseIcebergConnectorTest.testOptimizeTimePartitionedTable:5398->BaseIcebergConnectorTest.testOptimizeTimePartitionedTable:5439->AbstractTestQueryFramework.assertUpdate:410 » QueryFailed Failed to close manifest writer

these 2 tests are failing, so trying to check, if there is any relation with the current PR changes.

Rerun of the prb looks them good, could be the flaky one.

@sopel39 sopel39 merged commit 5ff71df into trinodb:master Jul 23, 2024
96 checks passed
@github-actions github-actions bot added this to the 453 milestone Jul 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

3 participants