Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minor: fix: Include FetchRel when producing LogicalPlan from Sort #13862

Merged
merged 3 commits into from
Dec 21, 2024

Conversation

robtandy
Copy link
Contributor

Which issue does this PR close?

Closes #13860

Rationale for this change

Explained in #13860

What changes are included in this PR?

Are these changes tested?

Included a test that fails before this change

Are there any user-facing changes?

to_substrait_plan will include a FetchRel when serializing a Sort that contains a fetch. It had previously missed this.

@robtandy
Copy link
Contributor Author

Per contribution guide
For new contributors a committer must first trigger the CI tasks. Please mention the members from committers list in the PR to help trigger the CI

Mentioning @alamb :)

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like an improvement to me. Thank you @robtandy

cc @Blizzara and @vbarua

}))
});

match sort.fetch {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Another pattern I have seen to avoid missing fields like this is

let Sort { input, offset } = sort ;
...

That way the compiler will tell you when you have not handled some field and you have to explicitly list it out

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making sure I follow:

You mean on line 364 destructuring the plan like

LogicalPlan::Sort(Sort {expr, input, fetch}) => {...

I didn't see that pattern followed in that file for the other LogicalPlan inners. I think its a good idea though

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment on lines 384 to 386
let count_mode = sort
.fetch
.map(|amount| {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

given you have the match sort.fetch { Some(_) => above, you could store that "some" and then use it here

match sort.fetch {
   Some(amount) => { 
      ...
      let count_mode = to_substrait_rex(.., (amount as i64), ...)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also I think rather than using to_substrait_rex, you could create the substrait expr directly, that might be nicer here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ty for the suggestions! Heading to dinner now and I can address these tomorrow morning eastern time.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incorporated both suggestions. Thank you @Blizzara

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love it -- thank you so much @robtandy and @Blizzara

🚀

@alamb alamb merged commit a50ed34 into apache:main Dec 21, 2024
24 checks passed
zhuqi-lucas pushed a commit to zhuqi-lucas/arrow-datafusion that referenced this pull request Dec 23, 2024
…ache#13862)

* include FetchRel when producing LogicalPlan from Sort

* add suggested test

* address review feedback
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Substrait roundtrip fails for Sort with a fetch
3 participants