Problem changing from normal to devel queue on Betzy

[TomasTorsvik]

@mvertens , @gold2718 , @monsieuralok , @mvdebolskiy
Hi,
I wonder, is there any way to change from the normal to the devel queue on Betzy without editing the env_batch.xml file by hand?
When running on devel the sbatch script should receive the directive --qos=devel, as mentioned in the documentation
https://noresm-docs.readthedocs.io/en/noresm2/configurations/platforms.html#queue-options-on-fram-and-on-betzy
and in issue #471 . Ideally, we should not edit xml files by hand, but I don't see how to do this through xmlchange. Is there some way to do this, in particular for noresm2.0.7?

[mvertens]

@TomasTorsvik - I believe that with the changes that @mvdebolsky brought into the latest cime tag - you should just be able to do an xmlchange JOB_QUEUE=devel and the right directives should be submitted. I can test this.

[JorgSchwinger]

Don't know if this helps, but for NorESM2 I used ./case.submit --batch-args="--qos=devel"

[mvdebolskiy]

you can specify --queue devel in your ./create_newcase. Also, doing ./xmlchange JOB_QUEUE will change job queues for both run and st_archive so the proper way would be ./xml change --subgroup case.run JOB_QUEUE this also applies when changing JOB_WALLCLOCK_TIME.

[TomasTorsvik]

@mvertens , @gold2718 , @mvdebolskiy , @monsieuralok
I tested it now with noresm2.0.7, but I still basically have the same problem, i.e. devel requires -q or --qos= while we only provide option for -p or --partition=. It's a bit confusing in the documentation from Sigma2, but it seems that -p and -q can be combined, and perhaps we should have a separate $JOB_QOS variable to set -q?

https://documentation.sigma2.no/jobs/submitting/queue_system_concepts.html
https://documentation.sigma2.no/jobs/choosing_job_types.html#job-types
https://documentation.sigma2.no/jobs/job_types/betzy_job_types.html#job-types-betzy

[gold2718]

Can you point to a case directory? Did you re-run checkout_externals after updating to the new release-noresm2.0.7, then start a new case?

[TomasTorsvik]

OK, my bad. Checked out the new release-noresm2.0.7, but forgot to run checkout externals ;)

By the way, the case directory is
/cluster/projects/nn9039k/people/tomast/2023-NorESM-workshop/cases/

[mvertens]

@TomasTorsvik - In your $CASEROOT wiht noresm2.0.7 - you will see the following in env_batch.xml:

    <directives queue="devel">
      <directive> --partition=normal</directive>
      <directive> --qos=devel</directive>
     </directive>

This should pick up the --qos=devel for your submission.