Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support heterogeneous workers #145

Open
mschubert opened this issue May 10, 2019 · 1 comment
Open

Support heterogeneous workers #145

mschubert opened this issue May 10, 2019 · 1 comment
Milestone

Comments

@mschubert
Copy link
Owner

Allow concatenation of different workers to a "pool."

Workflow logic is basically unchanged, because it does not matter where the workers are as long as they connect to the same socket.

@mschubert mschubert added this to the v1.0 milestone May 10, 2019
@bart1
Copy link

bart1 commented Sep 14, 2020

This seems useful and might address my needs. I run into a situation where I have difficulty using the cluster efficiently. On my cluster the smallest unit is a 16 processes and I cant run long processes on the submitting machines. This means I until now was able to run everything within one node (master and workers using multicore). This however produces a scalability problem if i want to go to multiple nodes. In that case I could run a SLURM setup but this means that on the master node I only run one process while I would still have space for 15 workers. Combining local and slurm workers might solve this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants