support LargeList for arrow_cast, support ScalarValue::LargeList
#8290
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
7 tasks
alamb
changed the title
LargeList for arrow_castLargeList for arrow_cast, support ScalarValue::LargeList
Weijun-H
force-pushed
the
scalar-largelist
branch
from
4f54551 to
991f3af
Compare
alamb
reviewed
Nov 22, 2023
| @@ -342,7 +346,38 @@ impl PartialOrd for ScalarValue { | |||
| None | |||
| } | |||
| } | |||
| (LargeList(arr1), LargeList(arr2)) => { | |||
| if arr1.data_type() == arr2.data_type() { | |||
There was a problem hiding this comment.
@Weijun-H can you try (in a follow on PR) to use the arrow lt kernels diectly on the list (rather than calling it on each element)? I can't remember if the kernel knows how to handle things
| if list_arr1.len() != list_arr2.len() { | ||
| return None; | ||
| } | ||
| for i in 0..list_arr1.len() { |
There was a problem hiding this comment.
I think the list lengths should always be 1, so I don't really understand why this is iterating (though I see it is just following the pattern of ListArray)
There was a problem hiding this comment.
Maybe you can clean it up in a follow on PR (by asserting the list is length 1)
There was a problem hiding this comment.
I will work on this on the next PR
There was a problem hiding this comment.
I think @jayzhan211 is working on this #8253
|
Thanks again @Weijun-H |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes #.
Rationale for this change
It is hard to test
LargeListin sqllogictest #8121, the convenient idea is to usearrow_cast()instead ofmake_arrayWhat changes are included in this PR?
LargeListforarrow_castScalarValue::LargeListScalarValue::new_large_listLargeListfor protobufAre these changes tested?
Are there any user-facing changes?