Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Figure out if batch work for the cluster workers is even faster #1

Open
balupton opened this issue Nov 30, 2015 · 0 comments
Open

Figure out if batch work for the cluster workers is even faster #1

balupton opened this issue Nov 30, 2015 · 0 comments

Comments

@balupton
Copy link
Member

Right now we send each task for the worker individually, this may incur a communication cost, perhaps we can batch the tasks or queue the tasks so that instead of sending each task to the worker individually, we send a batch of 10 at a time, or however many we have queued up while the worker was running, this may reduce communication cost and increase performance at the cost of additional complexity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant