You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Since iris 3.6, it is possible to use Dask distributed with iris. This is a great new feature that makes it possible to implement distributed computing (e.g. running a single recipe on multiple cluster nodes) in ESMValCore and ESMValTool. See #1714 for an example implementation. I would like to add this feature.
The text was updated successfully, but these errors were encountered:
After gaining some more experience with Dask I'm not sure if this is a good idea because
diagnostic tasks are likely to use more memory than workers have
for preprocessing tasks, metadata handling may also exceed the amount of memory that is available on a Dask worker, though this could probably be fixed
for preprocessing tasks a more fine-grained approach would parallelize much better
Since iris 3.6, it is possible to use Dask distributed with iris. This is a great new feature that makes it possible to implement distributed computing (e.g. running a single recipe on multiple cluster nodes) in ESMValCore and ESMValTool. See #1714 for an example implementation. I would like to add this feature.
The text was updated successfully, but these errors were encountered: