Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve performance by specializing mutable array data for each data type #6815

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

rluvaton
Copy link
Contributor

@rluvaton rluvaton commented Nov 29, 2024

Note

This is still work in progress, opening as draft to see if it's even wanted

Which issue does this PR close?

none

Rationale for this change

I think we can get better performance by specializing mutable array data for each data type and avoiding the dyn dispatch

What changes are included in this PR?

Creating mutable array data for each data type

Are there any user-facing changes?

yes

@github-actions github-actions bot added the arrow Changes to the arrow crate label Nov 29, 2024
@tustvold
Copy link
Contributor

tustvold commented Nov 29, 2024

I think if we want to go down this path it would be better to implement this at the array level in arrow-select, with a view to eventually removing MutableArrayData #2832 and better supporting other use-cases such as #6692.

I hope to find some time over the next few weeks to sketch out some designs

@rluvaton
Copy link
Contributor Author

I'm swamped with notification from other open source projects so I (with high possibility) will miss the notification, if so, please don't hesitate to ping me on X

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants