You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
would it make sense, and be acceptable to the team, to have a sub-repo within pyjanitor-devs (like pyjviz), but focused on performance within a lower level language (cython, rust, c, ...). that way it is not conflated with core pyjanitor, and yet we can steal perf. I'm thinking of it, especially now that we are supporting polars, there are some cases where we may have to write extensions to get performance boost within polars (an example is strip_accents within clean_names does not take advantage of Polars' built-in functions, because there is no support for it - with this proposal, we could potentially write it in rust and pass it to polars engine for more performance). Of course, we could also use it for Pandas (my target at the moment is conditional_join, but this could potentially help with other functions).
The text was updated successfully, but these errors were encountered:
samukweku
changed the title
sub-repo for lower level performance optimisations
sub-repo for performance optimisations in a lower level language
Jun 28, 2024
would it make sense, and be acceptable to the team, to have a sub-repo within pyjanitor-devs (like pyjviz), but focused on performance within a lower level language (cython, rust, c, ...). that way it is not conflated with core pyjanitor, and yet we can steal perf. I'm thinking of it, especially now that we are supporting polars, there are some cases where we may have to write extensions to get performance boost within polars (an example is
strip_accents
withinclean_names
does not take advantage of Polars' built-in functions, because there is no support for it - with this proposal, we could potentially write it in rust and pass it to polars engine for more performance). Of course, we could also use it for Pandas (my target at the moment isconditional_join
, but this could potentially help with other functions).@pyjanitor-devs/core-devs @ericmjl
The text was updated successfully, but these errors were encountered: