Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CYLC_SHARE_CYCLE_DIR to task script. #6117

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

wxtim
Copy link
Member

@wxtim wxtim commented May 31, 2024

Closes #6098

Check List

  • I have read CONTRIBUTING.md and added my name as a Code Contributor.
  • Contains logically grouped changes (else tidy your branch by rebase).
  • Does not contain off-topic changes (use other PRs for other changes).
  • Applied any dependency changes to both setup.cfg (and conda-environment.yml if present).
  • Tests are included (or explain why tests are not needed).
  • CHANGES.md entry included if this is a change that can affect users
  • Cylc-Doc pull request opened if required at cylc/cylc-doc/pull/XXXX.
  • If this is a bug fix, PR should be raised against the relevant ?.?.x branch.

@wxtim wxtim self-assigned this May 31, 2024
@wxtim wxtim marked this pull request as draft May 31, 2024 09:16
@wxtim wxtim added the small label May 31, 2024
@wxtim wxtim added this to the 8.3.1 milestone May 31, 2024
@wxtim wxtim force-pushed the feat.share_cycle_cylce_directory branch from b683ee3 to 02e0c65 Compare May 31, 2024 09:18
@wxtim wxtim requested a review from oliver-sanders May 31, 2024 10:12
@wxtim wxtim marked this pull request as ready for review May 31, 2024 10:12
@ColemanTom
Copy link
Contributor

Just as a note. If people don't use rose task-env, nor the cycle directory, and never have, then this will be creating a folder that they now have to cleanup else risk leaving lots of empty folders on disk, using up potential inode quotas or having too many files on lustre in a single folder, causing performance degredation.

@oliver-sanders
Copy link
Member

Yes, this will create one folder per-cycle. This doesn't seem particularly concerning to me given that we are already creating one folder and four-six files per-job. As a rule of thumb count(jobs) >> count(cycles) so I had not considered that this would be flagged as a performance issue.

However, if you have an existing housekeeping arrangement for log/job/<cycle>, then this might force inclusion of share/cycle/<cycle> in that arrangement (note rose_prune can manage this, one day Cylc may be able to do this itself). WDYT?

@elliotfontaine
Copy link

If I may put my two cents in, this would be really great! I use Rose anyway for the rose-suite.conf file, but this should be Cylc functionality as @oliver-sanders said in May. Right now I have $(eval rose task-env) in my [[root]] task, which is kinda weird.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

share/cycle/<cycle> directory
4 participants