Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support --qos as a resource #27

Closed
wants to merge 2 commits into from
Closed

feat: support --qos as a resource #27

wants to merge 2 commits into from

Conversation

kdm9
Copy link

@kdm9 kdm9 commented Jan 29, 2024

A cluster I am using requires setting --qos to enable jobs with longer runtimes. it would be convenient to be directly able to use slurm_qos as a resource, like one can do for partition & account, so I've implemented that. Also fixed some typos in the docs.

best
K

@kdm9 kdm9 changed the title support --qos as a resource feat: support --qos as a resource Jan 29, 2024
@fgvieira
Copy link
Contributor

fgvieira commented Feb 2, 2024

Can't you use slurm_extra to pass the qos?

@cmeesters
Copy link
Member

Essentially, this is all covered by slurm_extra. However, the documentation needs an update in that regard (and I do not seem to get to it all too soon) and there has been a bug.

Please update to snakemake v8.4.3 or higher, that should fix working with slurm_extra. Yet, there is another bug, see PR #2627 and the discussion to #2555.

Note, that the idea behind this executor is to provide all relevant submission flags, but to catch cluster relevant flags, like qos (where some admins hide their accelerators, instead of gres) or flags introduced by spank plugins or all the nifty detail flags, sbatch provides with slurm_extra. If we were to include all flags explicetly, we would have to include a hundred ifs in the code.

@kdm9
Copy link
Author

kdm9 commented Feb 8, 2024

How about something like:

for resource, value in job.resources.items:
    if not resource.startswith("slurm_"):
        continue
    slurmarg = resource.replace("slurm_", "")
    args += f" --{slurmarg}={value}"

In other words, to submit with sbatch --arg=value, provide "slurm_arg=value" as a resource.

That should handle partition as it happens now (perhaps updating job.resources["slurm_partition"] using the existing logic), and would allow users direct control over the submission command.

@cmeesters
Copy link
Member

I am closing this pull request: The current release is working with slurm_extra as intended, offering to set qos.

@cmeesters cmeesters closed this Feb 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants