Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sub-repo for performance optimisations in a lower level language #1381

Open
samukweku opened this issue Jun 27, 2024 · 0 comments
Open

sub-repo for performance optimisations in a lower level language #1381

samukweku opened this issue Jun 27, 2024 · 0 comments

Comments

@samukweku
Copy link
Collaborator

samukweku commented Jun 27, 2024

would it make sense, and be acceptable to the team, to have a sub-repo within pyjanitor-devs (like pyjviz), but focused on performance within a lower level language (cython, rust, c, ...). that way it is not conflated with core pyjanitor, and yet we can steal perf. I'm thinking of it, especially now that we are supporting polars, there are some cases where we may have to write extensions to get performance boost within polars (an example is strip_accents within clean_names does not take advantage of Polars' built-in functions, because there is no support for it - with this proposal, we could potentially write it in rust and pass it to polars engine for more performance). Of course, we could also use it for Pandas (my target at the moment is conditional_join, but this could potentially help with other functions).

@pyjanitor-devs/core-devs @ericmjl

@samukweku samukweku changed the title sub-repo for lower level performance optimisations sub-repo for performance optimisations in a lower level language Jun 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant