Add CYLC_SHARE_CYCLE_DIR to task script.

[wxtim]

Closes #6098

Check List

• I have read CONTRIBUTING.md and added my name as a Code Contributor.
• Contains logically grouped changes (else tidy your branch by rebase).
• Does not contain off-topic changes (use other PRs for other changes).
• Applied any dependency changes to both setup.cfg (and conda-environment.yml if present).
• Tests are included (or explain why tests are not needed).
• CHANGES.md entry included if this is a change that can affect users
• Cylc-Doc pull request opened if required at cylc/cylc-doc/pull/736.
• If this is a bug fix, PR should be raised against the relevant ?.?.x branch.

[ColemanTom]

Just as a note. If people don't use rose task-env, nor the cycle directory, and never have, then this will be creating a folder that they now have to cleanup else risk leaving lots of empty folders on disk, using up potential inode quotas or having too many files on lustre in a single folder, causing performance degredation.

[oliver-sanders]

Yes, this will create one folder per-cycle. This doesn't seem particularly concerning to me given that we are already creating one folder and four-six files per-job. As a rule of thumb count(jobs) >> count(cycles) so I had not considered that this would be flagged as a performance issue.

However, if you have an existing housekeeping arrangement for log/job/<cycle>, then this might force inclusion of share/cycle/<cycle> in that arrangement (note rose_prune can manage this, one day Cylc may be able to do this itself). WDYT?

[elliotfontaine]

If I may put my two cents in, this would be really great! I use Rose anyway for the rose-suite.conf file, but this should be Cylc functionality as @oliver-sanders said in May. Right now I have $(eval rose task-env) in my [[root]] task, which is kinda weird.

[hjoliver]

IMO this is good.

• one additional folder per cycle is not much, compared to the per-task folder/file count as Oliver noted
• any cycling workflow is very likely to need to a per-cycle dir like this
  • (and for non-cycling workflows this will only create one folder)
• housekeeping is pretty easy if needed

[MetRonnie]

Kicking tests as they last ran 8 months ago

[MetRonnie]

Question on the name of the environment variable:

• CYLC_SHARE_CYCLE_DIR (✔️ short)
• CYLC_WORKFLOW_SHARE_CYCLE_DIR (✔️ consistent with other env vars)

[MetRonnie]

This new env var should be added to https://github.com/cylc/cylc-doc/blob/master/src/reference/job-script-vars/var-list.txt

[wxtim]

This new env var should be added to https://github.com/cylc/cylc-doc/blob/master/src/reference/job-script-vars/var-list.txt

Cylc doc PR already exists, and is linked in the heading - though I didn't change the XXX in the link label, so you might have missed it. (Also PR had the wrong variable name, now fixed)

CYLC_WORKFLOW_SHARE_CYCLE_DIR (✔️ consistent with other env vars)

To my mind that would imply a variable constant across all cycles and tasks in the workflow, which this isn't!

[MetRonnie]

Yes you're right, CYLC_TASK_SHARE_CYCLE_DIR would be consistent with other vars

[wxtim]

Yes you're right, CYLC_TASK_SHARE_CYCLE_DIR would be consistent with other vars

No, because that would imply it's unique on a per task basis.

[MetRonnie]

There are other CYLC_TASK_ vars which apply on a per-task basis without being unique to tasks. Case in point: CYLC_TASK_CYCLE_POINT

[MetRonnie]

Happy unless there is a consensus to call it CYLC_TASK_SHARE_CYCLE_DIR