Upcoming changes on stashing

[khsrali]

I plan a series of development that I'm bringing to stashing.
To avoid 3-4 K line of PR, which would be hard to review I choose doing it in several PRs, that each individual brings a piece. The stashing so far, is only supported as an attached step to a generic calcjob.

The upcoming changes are as following:

• Support for compression formats during stashing, TARGZ ,etc.
  • Transport: feat compress & extract methods #6743
  • RemoteStashCompressedData: new data class providing compression modes when stashing #6746
• Stashing as a separate calcjob is useful, especially when performed after completing a generic calcjob.
  • StashCalculation: a new CalcJob plugin #6772
• Enable stashing through a custom script, either as a standalone calcjob or as an attached step within a generic calcjob.
  • RemoteStashCustomData: a class for stashed data via custom script, and the implementation of stashing via custom script #6777
• Develop an unstashing calcjob that supports both restoring data to its original location and generating new data nodes.
  • TODO. I'll wait for the design until all above PRs are reviewed and merged.
• Enable re-stashing to accommodate storage movements without requiring un-stashing.
  • TODO. I'll wait for the design until all above PRs are reviewed and merged.
• A supporting verdi cli, perhaps verdi stash that run those calcjobs.
  • Not confirmed.

@agoscinski for your info --since you showed interest to review these changes 😉

[agoscinski]

What are your goals regarding this issue and release v2.7.0?

[khsrali]

sorry @agoscinski, somehow I overlooked this issue, didn't see your question.

So, yeah.. let's see how fast it goes. I'd say whetever we merge from top to bottom we include, the rest can wait. 🤷‍♂
If somebody urgently would need this, we can always make another release.