Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

multithreaded optimization of label-aggregation #59

Open
jesteria opened this issue Apr 13, 2021 · 1 comment
Open

multithreaded optimization of label-aggregation #59

jesteria opened this issue Apr 13, 2021 · 1 comment
Labels
optimization Smaller/faster/better question Further information is requested

Comments

@jesteria
Copy link
Collaborator

Label-aggregation performance was hugely improved by #54. However, it remains a single-threaded affair. It should be investigated whether multithreading/multiprocessing can further improve the performance of this step, and if so, implemented.
@jesteria jesteria added question Further information is requested optimization Smaller/faster/better labels Apr 13, 2021
@jesteria jesteria mentioned this issue Apr 13, 2021
Merged
@JordanHolland
Copy link
Collaborator

This seems like a tricky balance between shuffling data and actually aggregating, which could depend on the size of each individual nPrint. If we get it right, we should see improvement, but it's really hard to gauge just how much.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
optimization Smaller/faster/better question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jesteria @JordanHolland