fix: Enable Parquet E2E filter tests on decimal type #11602
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
the
CLA Signed
rui-mo
changed the title
✅ Deploy Preview for meta-velox canceled.
|
rui-mo
force-pushed
the
wip_part_decimal
branch
from
fe3b410 to
98285dd
Compare
rui-mo
force-pushed
the
wip_part_decimal
branch
2 times, most recently
from
0ff89b3 to
bcdb766
Compare
rui-mo
commented
Nov 20, 2024
There was a problem hiding this comment.
@Yuhta @majetideepak Would you like to take a review? Thanks.
| @@ -316,6 +316,9 @@ bool testFilter( | |||
| return true; | |||
| } | |||
| if (type->isDecimal()) { | |||
| // The min and max value in the metadata for decimal type in Parquet can be | |||
| // stored in different physical types, including int32, int64 and | |||
| // fixed_len_byte_array. The loading of them is not supported in Metadata. | |||
There was a problem hiding this comment.
I took a look at the support of row group skip for Parquet decimal and am glad to follow up in a separate PR if you think it is needed.
Yuhta
approved these changes
Nov 20, 2024
Yuhta
approved these changes
Nov 20, 2024
Yuhta
added
the
ready-to-merge
|
@kagamiori has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
1 similar comment
|
@kagamiori has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@kagamiori merged this pull request in 1b75704. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
can be generated.
on column statistics for decimal type is not currently supported.