Merge branch 'NCAR:main' into seaice3

.github/PULL_REQUEST_TEMPLATE/pull_request_template.md → .github/PULL_REQUEST_TEMPLATE.md

@@ -1,6 +1,6 @@
 ### All Submissions:
 
-* [ ] Have you followed the guidelines in our Contributer's Guide](https://github.com/NCAR/CUPiD/wiki/Contributor's-Guide) (including the `pre-commit` check)?
+* [ ] Have you followed the guidelines in our [Contributor's Guide](https://github.com/NCAR/CUPiD/wiki/Contributor's-Guide) (including the `pre-commit` check)?
 * [ ] Have you checked to ensure there aren't other open [Pull Requests](../../../pulls) for the same update/change?
 
 <!-- You can erase any parts of this template not applicable to your Pull Request. -->
@@ -13,5 +13,4 @@
 ### Changes to Core Features:
 
 * [ ] Have you added an explanation of what your changes do and why you'd like us to include them?
-* [ ] Have you written new tests for your core changes, as applicable?
-* [ ] Have you successfully ran tests with your changes locally?
+* [ ] Have you successfully tested your changes locally?

.github/workflows/tasklist_completed.yml

@@ -0,0 +1,14 @@
+name: 'PR Tasks Completed Check'
+on:
+  pull_request:
+    types: [opened, edited]
+
+jobs:
+  task-check:
+    runs-on: ubuntu-latest
+    permissions:
+      checks: write
+    steps:
+      - uses: kentaro-m/[email protected]
+        with:
+          repo-token: "${{ secrets.GITHUB_TOKEN }}"

.gitignore

@@ -12,9 +12,8 @@ cupid.egg-info
 # Text editor temp files
 **/*.swp
 
-externals/
-
 # Documentation
 /docs/_build/
 /docs/README.md
 /docs/NCAR_tips.md
+/docs/contributors.md

.gitmodules

@@ -1,3 +1,6 @@
-[submodule "manage_externals"]
-	path = manage_externals
-	url = https://github.com/ESMCI/manage_externals.git
+[submodule "externals/ADF"]
+	path = externals/ADF
+	url = https://github.com/NCAR/ADF.git
+[submodule "externals/mom6-tools"]
+	path = externals/mom6-tools
+	url = https://github.com/NCAR/mom6-tools.git

README.md

@@ -13,18 +13,17 @@ This collaborative effort aims to simplify the user experience of running diagno
 To install CUPiD, you need to check out the code and then set up a few environments.
 The initial examples have hard-coded paths that require you to be on `casper`.
 
-The code relies on submodules to install `manage_externals` and then uses `manage_externals` for a few packages that are still being developed,
-so the `git clone` process is a little more complicated than usual:
+The code relies on submodules to install a few packages that are still being developed,
+so the `git clone` process requires `--recurse-submodules`:
 
 ``` bash
 $ git clone --recurse-submodules https://github.com/NCAR/CUPiD.git
-$ cd CUPiD
-$ ./manage_externals/checkout_externals
 ```
 
-Then build the necessary conda environments with
+Then `cd` into the `CUPiD` directory and build the necessary conda environments with
 
 ``` bash
+$ cd CUPiD
 $ mamba env create -f environments/dev-environment.yml
 $ conda activate cupid-dev
 $ which cupid-run
@@ -37,7 +36,7 @@ Notes:
 It still feels slower than running `mamba` directly, hence the recommendation to install with `mamba env create` rather than `conda env create`.
 If you do not have `mamba` installed, you can still use `conda`... it will just be significantly slower.
 (To see what version of conda you have installed, run `conda --version`.)
-1. If `./manage_externals/checkout_externals` is not found, run `git submodule update --init` to clone the submodule.
+1. If the subdirectories in `externals/` are all empty, run `git submodule update --init` to clone the submodules.
 1. If `which cupid-run` returned the error `which: no cupid-run in ($PATH)`, then please run the following:
 
    ``` bash
@@ -67,6 +66,14 @@ $ cupid-build  # Will build HTML from Jupyter Book
 After the last step is finished, you can use Jupyter to view generated notebooks in `${CUPID_ROOT}/examples/coupled-model/computed_notebooks/quick-run`
 or you can view `${CUPID_ROOT}/examples/coupled-model/computed_notebooks/quick-run/_build/html/index.html` in a web browser.
 
+Notes:
+
+1. Occasionally users report the following error the first time they run CUPiD: `Environment cupid-analysis specified for <YOUR-NOTEBOOK>.ipynb could not be found`. The fix for this is the following:
+   ``` bash
+   $ conda activate cupid-analysis
+   (cupid-analysis) $ python -m ipykernel install --user --name=cupid-analysis
+   ```
+
 Furthermore, to clear the `computed_notebooks` folder which was generated by the `cupid-run` and `cupid-build` commands, you can run the following command:
 
 ``` bash

contributors.md

@@ -0,0 +1,37 @@
+# Alphabetical list of all contributors
+ - Dave Bailey (NCAR), [@dabail10](https://github.com/dabail10)
+ - Ingrid Carlson (Carroll College), [@ingridc2051](https://github.com/ingridc2051)
+ - Dani Coleman (NCAR), [@bitterbark](https://github.com/bitterbark)
+ - Cameron Cummins (U Texas), [@AgentOxygen](https://github.com/AgentOxygen)
+ - Katie Dagon (NCAR), [@katiedagon](https://github.com/katiedagon)
+ - Anna-Lena Deppenmeier (NCAR; now University of Liverpool), [@ALDepp](https://github.com/ALDepp)
+ - Brian Dobbins (NCAR), [@briandobbins](https://github.com/briandobbins)
+ - Orhan Eroglu (NCAR), [@erogluorhan](https://github.com/erogluorhan)
+ - John Fasullo (NCAR)
+ - Katelyn FitzGerald (NCAR), [@kafitzgerald](https://github.com/kafitzgerald)
+ - Meg Fowler (NCAR), [@megandevlan](https://github.com/megandevlan)
+ - Cecile Hannay (NCAR), [@cecilehannay](https://github.com/cecilehannay)
+ - Teagan King (NCAR), [@TeaganKing](https://github.com/TeaganKing)
+ - Shivani Kumar (U Michigan), [@shivaniikum](https://github.com/shivaniikum)
+ - Hilary Lam (U Michigan)
+ - Dave Lawrence (NCAR), [@dlawrenncar](https://github.com/dlawrenncar)
+ - Sam Levis (NCAR), [@slevis-lmwg](https://github.com/slevis-lmwg)
+ - Mike Levy (NCAR), [@mnlevy1981](https://github.com/mnlevy1981)
+ - Gustavo Marques (NCAR), [@gustavo-marques](https://github.com/gustavo-marques)
+ - Christina McCluskey (NCAR), [@cmcclus](https://github.com/cmcclus)
+ - Brian Medeiros (NCAR), [@brianpm](https://github.com/brianpm)
+ - Naoki Mizukami (NCAR), [@nmizukami](https://github.com/nmizukami)
+ - Jesse Nusbaumer (NCAR), [@nusbaume](https://github.com/nusbaume)
+ - Keith Oleson (NCAR), [@olyson](https://github.com/olyson)
+ - Adam Phillips (NCAR), [@phillips-ad](https://github.com/phillips-ad)
+ - Justin Richling (NCAR), [@justin-richling](https://github.com/justin-richling)
+ - Lev Romashkov (NCAR; now Stanford University), [@rmshkv](https://github.com/rmshkv)
+ - Bill Sacks (NCAR), [@billsacks](https://github.com/billsacks)
+ - Isla Simpson (NCAR), [@islasimpson](https://github.com/islasimpson)
+ - Gary Strand (NCAR), [@strandwg](https://github.com/strandwg)
+ - Kate Thayer-Calder (NCAR), [@Katetc](https://github.com/Katetc)
+ - Will Wieder (NCAR), [@wwieder](https://github.com/wwieder)
+ - Anissa Zacharias (NCAR), [@anissa111](https://github.com/anissa111)
+ - Feng Zhu (NCAR), [@fzhu2e](https://github.com/fzhu2e)
+
+Note: If you have contributed to the project, please feel free to add yourself to this list by updating [contributors.md](https://github.com/NCAR/CUPiD/blob/main/contributors.md).

cupid/clear.py

@@ -52,8 +52,8 @@ def clear(config_path):
     Args: CONFIG_PATH - The path to the configuration file.
 
     """
-
+    logger = cupid.util.setup_logging(config_path)
     run_dir = read_config_file(config_path)
     # Delete the "computed_notebooks" folder and all the contents inside of it
     shutil.rmtree(run_dir)
-    print(f"All contents in {run_dir} have been cleared.")
+    logger.info(f"All contents in {run_dir} have been cleared.")

cupid/run.py

@@ -23,7 +23,6 @@
 from __future__ import annotations
 
 import os
-import warnings
 
 import click
 import intake
@@ -74,6 +73,7 @@ def run(
     # Get control structure
     control = cupid.util.get_control_dict(config_path)
     cupid.util.setup_book(config_path)
+    logger = cupid.util.setup_logging(config_path)
 
     component_options = {
         "atm": atmosphere,
@@ -110,16 +110,67 @@ def run(
 
         for component, comp_bool in component_options.items():
             if comp_bool:
+
+                # set time series input and output directory:
+                # -----
+                if isinstance(timeseries_params["case_name"], list):
+                    ts_input_dirs = []
+                    for cname in timeseries_params["case_name"]:
+                        ts_input_dirs.append(global_params["CESM_output_dir"]+"/"+cname+f"/{component}/hist/")
+                else:
+                    ts_input_dirs = [
+                        global_params["CESM_output_dir"] + "/" +
+                        timeseries_params["case_name"] + f"/{component}/hist/",
+                    ]
+
+                if "ts_output_dir" in timeseries_params:
+                    if isinstance(timeseries_params["ts_output_dir"], list):
+                        ts_output_dirs = []
+                        for ts_outdir in timeseries_params["ts_output_dir"]:
+                            ts_output_dirs.append([
+                                os.path.join(
+                                        ts_outdir,
+                                        f"{component}", "proc", "tseries",
+                                ),
+                            ])
+                    else:
+                        ts_output_dirs = [
+                            os.path.join(
+                                    timeseries_params["ts_output_dir"],
+                                    f"{component}", "proc", "tseries",
+                            ),
+                        ]
+                else:
+                    if isinstance(timeseries_params["case_name"], list):
+                        ts_output_dirs = []
+                        for cname in timeseries_params["case_name"]:
+                            ts_output_dirs.append(
+                                os.path.join(
+                                        global_params["CESM_output_dir"],
+                                        cname,
+                                        f"{component}", "proc", "tseries",
+                                ),
+                            )
+                    else:
+                        ts_output_dirs = [
+                            os.path.join(
+                                    global_params["CESM_output_dir"],
+                                    timeseries_params["case_name"],
+                                    f"{component}", "proc", "tseries",
+                            ),
+                        ]
+                # -----
+
                 # fmt: off
                 # pylint: disable=line-too-long
                 cupid.timeseries.create_time_series(
                     component,
                     timeseries_params[component]["vars"],
                     timeseries_params[component]["derive_vars"],
-                    [timeseries_params["case_name"]],
+                    timeseries_params["case_name"],
                     timeseries_params[component]["hist_str"],
-                    [global_params["CESM_output_dir"]+"/"+timeseries_params["case_name"]+f"/{component}/hist/"],
-                    [global_params["CESM_output_dir"]+"/"+timeseries_params["case_name"]+f"/{component}/proc/tseries/"],
+                    ts_input_dirs,
+                    ts_output_dirs,
                     # Note that timeseries output will eventually go in
                     #   /glade/derecho/scratch/${USER}/archive/${CASE}/${component}/proc/tseries/
                     timeseries_params["ts_done"],
@@ -129,6 +180,7 @@ def run(
                     timeseries_params[component]["level"],
                     num_procs,
                     serial,
+                    logger,
                 )
                 # fmt: on
                 # pylint: enable=line-too-long
@@ -194,7 +246,7 @@ def run(
                     all_nbs[nb]["nb_path_root"] = nb_path_root + "/" + comp_name
                     all_nbs[nb]["output_dir"] = output_dir + "/" + comp_name
             elif comp_bool and not all:
-                warnings.warn(
+                logger.warning(
                     f"No notebooks for {comp_name} component specified in config file.",
                 )
 
@@ -203,7 +255,7 @@ def run(
         for nb, info in all_nbs.copy().items():
             if not control["env_check"][info["kernel_name"]]:
                 bad_env = info["kernel_name"]
-                warnings.warn(
+                logger.warning(
                     f"Environment {bad_env} specified for {nb}.ipynb could not be found;" +
                     f" {nb}.ipynb will not be run." +
                     "See README.md for environment installation instructions.",
@@ -237,7 +289,7 @@ def run(
                     all_scripts[script] = info
                     all_scripts[script]["nb_path_root"] = nb_path_root + "/" + comp_name
             elif comp_bool and not all:
-                warnings.warn(
+                logger.warning(
                     f"No scripts for {comp_name} component specified in config file.",
                 )
 
@@ -246,7 +298,7 @@ def run(
         for script, info in all_scripts.copy().items():
             if not control["env_check"][info["kernel_name"]]:
                 bad_env = info["kernel_name"]
-                warnings.warn(
+                logger.warning(
                     f"Environment {bad_env} specified for {script}.py could not be found;" +
                     f"{script}.py will not be run.",
                 )