This repository contains a collection of post-processed mesoscale cloud classifications.
The dependencies are listed in environment.yml
and can be installed e.g. with conda or mamba:
mamba env create -f environment.yml
or directly into an existing python environment with
mamba install python intake-xarray xarray s3fs "intake<2.0.0"
mamba can be downloaded at https://github.com/conda-forge/miniforge/releases/
To access the mesoscale cloud morphology datasets, the following lines are sufficient (after installing any dependencies):
import intake
cat = intake.open_catalog("https://raw.githubusercontent.com/ISSI-CONSTRAIN/meso-morphs/main/catalog/catalog.yaml")
ds = cat.SGFF.to_dask()
Further catalog entries, like those referencing the MCC
and MEASURES
dataset, can be listed with
list(cat)
In case the dataset will be handled in a software environment different to python, it might be easiest to store a local copy of the dataset by running the above mentioned python instructions and save the datasets as netCDF files with:
ds.to_netcdf("SGFF_classifications.nc")
- To reproduce the workflow, access to the UW olympus cluster is needed
- Please set your olympus username as environment variable
export SSH_USERNAME=myname
dvc repro
downloads the original files and updates the output files- To push files to remote, please
export AWS_ACCESS_KEY_ID=${API Key}
andexport AWS_SECRET_ACCESS_KEY='mysecret'
- Only files in the output directory should be pushed with
dvc push --remote aws data/output/Daily*