Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support 'LargeList' in array_pop_front and array_pop_back #8569

Merged
merged 6 commits into from
Dec 28, 2023

Conversation

Weijun-H
Copy link
Member

@Weijun-H Weijun-H commented Dec 17, 2023

Which issue does this PR close?

Parts. #8185

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@github-actions github-actions bot added physical-expr Physical Expressions sqllogictest SQL Logic Tests (.slt) labels Dec 17, 2023
@Weijun-H Weijun-H mentioned this pull request Dec 17, 2023
19 tasks
Copy link
Contributor

@jayzhan211 jayzhan211 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM

let array = as_large_list_array(&args[0])?;
general_pop_list::<i64>(array, false)
}
_ => exec_err!(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comparing to
https://github.com/apache/arrow-datafusion/pull/8571/files#diff-48cc9cf1bfdb0214a9f625b384d1c4fd5967a9da61e8f22a5dc1c4c5800563b4R1301
no data types check, and the output error type mismatches, we may want to make array functions to have consistent codebase?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't get your point. What do you mean by no data type check? Because here are largelist and list type checks. @comphead

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for confusion,
what I was talking is

check_datatypes("array_positions", &[arr.values(), element])?;

which you do for array_positions method, but not doing for here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In array_positions, there are two arguments, array and element, which should be same data type. While for pop functions, they only accept one argument.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that is true, i missed that part. thanks @Weijun-H
what about output error types? in array positions you return not_impl and in this PR its exec_err, but likely the conditions looks same and the error type should also be the same?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, they should be the same, as exec_err . I will change them in a follow pr.

where
i64: TryInto<O>,
{
let (from_array, to_array) = if from_front {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it better we just split this to two function? Since they are so different, without checking true for pop front or pop back may also improve readability?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is better for readability and maintenance.

@Weijun-H Weijun-H force-pushed the support-largelist-pop branch from 034c50f to 4252f3f Compare December 27, 2023 11:42
@Weijun-H Weijun-H requested a review from comphead December 27, 2023 11:44
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @Weijun-H and @jayzhan211

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
physical-expr Physical Expressions sqllogictest SQL Logic Tests (.slt)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants