Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Drop nbscuid name #39

Merged
merged 2 commits into from
Jan 9, 2024
Merged

Drop nbscuid name #39

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7dd3a3e
Rename nbscuid -> cupid
mnlevy1981 Jan 9, 2024
81c2ccb
Update .gitignore
mnlevy1981 Jan 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
# nbscuid-built files
# cupid-built files
**/computed_notebooks/**
**/cache_data_path/**
**/cache_metadata_path/**
**/temp_data/**
**/ncfiles/**
**/test_product.txt
nbscuid.egg-info
cupid.egg-info
**/.ipynb_checkpoints
**/__pycache__

Expand Down
22 changes: 11 additions & 11 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,21 +30,21 @@ $ ./manage_externals/checkout_externals
Then build the necessary conda environments with

```
$ conda env create -f nbscuid/dev-environment.yml
$ conda activate nbscuid-dev
$ which nbscuid-run
$ conda env create -f mom6-environment.yml
$ conda env create -f cupid/dev-environment.yml
$ conda activate cupid-dev
$ which cupid-run
$ conda env create -f cupid-analysis.yml
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a clear new environment name, thanks for replacing env with analysis, too.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I figured if we were moving to a single environment for all diagnostics we should have a generic name :) Seeing this block of text, though, I wonder if the file name should be analysis-environment.yml

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like -environment is redundant? And then at least if people are using the environment on a machine with lots of additional environments it may help to know that it's the CUPiD one?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The environment could stay cupid-analysis if the filename changed, and then it would match dev-environment.yml. I'll start a new issue ticket, though, because I think we also want an envs/ directory that contains both environment files rather than having one in cupid/ and the other in the top level

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, seems reasonable to me! Thanks Mike!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more thought, actually - I feel like I've seen the standard for environment spec YAML files be to always call the file "environment.yml", and I specifically made the choice to add a descriptor to these ones for clarity, but given that we also have various config YAML files floating around between the different component diagnostics and CUPiD usecases, keeping "environment" might actually be good to lessen confusion.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(For additional context - the name of the environment is specified in the file and isn't linked to the yml file name)

```

Notes:

1. `conda` now defaults to using `mamba` to solve environments; the `mom6-environment.yml` environment is complicated, so older versions of `conda` should be updated (`conda update -n base conda`) or you should use `mamba` instead.
1. `conda` now defaults to using `mamba` to solve environments; the `cupid-analysis.yml` environment is complicated, so older versions of `conda` should be updated (`conda update -n base conda`) or you should use `mamba` instead.
1. If `./manage_externals/checkout_externals` is not found, run `git submodule update --init` to clone the submodule.
1. If `which nbscuid-run` returned the error `which: no nbscuid-run in ($PATH)`, then please run the following:
1. If `which cupid-run` returned the error `which: no cupid-run in ($PATH)`, then please run the following:

```
$ conda activate nbscuid-dev
$ pip install -e . # installs nbscuid
$ conda activate cupid-dev
$ pip install -e . # installs cupid
```

## Running
Expand All @@ -53,10 +53,10 @@ CUPiD currently provides two examples for generating diagnostics.
To test the package out, try to run `examples/adf-mom6`:

```
$ conda activate nbscuid-dev
$ conda activate cupid-dev
$ cd examples/adf-mom6
$ nbscuid-run config.yml
$ nbscuid-build config.yml # Will build HTML from Jupyter Book
$ cupid-run config.yml
$ cupid-build config.yml # Will build HTML from Jupyter Book
```

After the last step is finished, you can use Jupyter to view generated notebooks in `${CUPID_ROOT}/examples/adf-mom6/computed_notebooks/adf-quick-run`
Expand Down
2 changes: 1 addition & 1 deletion mom6-environment.yml → cupid-analysis.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: mom6-tools-nbscuid
name: cupid-analysis
channels:
- conda-forge
- nodefaults
Expand Down
0 nbscuid/__init__.py → cupid/__init__.py
View file
Open in desktop
File renamed without changes.
6 changes: 3 additions & 3 deletions nbscuid/_jupyter-book-config-defaults.yml → cupid/_jupyter-book-config-defaults.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

#######################################################################################
# Book settings
title: nbscuid default book
title: cupid default book
author: none
copyright: '2023'

Expand All @@ -18,7 +18,7 @@ execute:
latex:
latex_engine: pdflatex
latex_documents:
targetname: nbscuid-default.tex
targetname: cupid-default.tex

# Add a bibtex file so that we can create citations
# bibtex_bibfiles:
Expand All @@ -31,7 +31,7 @@ sphinx:

# Information about where the book exists on the web
# repository:
# url: https://github.com/rmshkv/nbscuid
# url: https://github.com/rmshkv/cupid
# path_to_book: notebooks
# branch: main

Expand Down
2 changes: 1 addition & 1 deletion nbscuid/build.py → cupid/build.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

def build():
"""
Build a Jupyter book based on the TOC in config.yml. Called by `nbscuid-build`.
Build a Jupyter book based on the TOC in config.yml. Called by `cupid-build`.

Args:
none
Expand Down
4 changes: 2 additions & 2 deletions nbscuid/dev-environment.yml → cupid/dev-environment.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
name: nbscuid-dev
name: cupid-dev
dependencies:
- python==3.11.4
- pip
- pip:
- -e ../
- -e ../
0 nbscuid/quickstart.py → cupid/quickstart.py
View file
Open in desktop
File renamed without changes.
0 nbscuid/read.py → cupid/read.py
View file
Open in desktop
File renamed without changes.
18 changes: 9 additions & 9 deletions nbscuid/run.py → cupid/run.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
from glob import glob
import papermill as pm
import intake
import nbscuid.util
import cupid.util
import sys
from dask.distributed import Client
import dask
Expand All @@ -13,7 +13,7 @@

def run():
"""
Main engine to set up running all the notebooks. Called by `nbscuid-run`.
Main engine to set up running all the notebooks. Called by `cupid-run`.

Args:
none
Expand All @@ -24,8 +24,8 @@ def run():

# Get control structure
config_path = str(sys.argv[1])
control = nbscuid.util.get_control_dict(config_path)
nbscuid.util.setup_book(config_path)
control = cupid.util.get_control_dict(config_path)
cupid.util.setup_book(config_path)

# Grab paths

Expand Down Expand Up @@ -88,10 +88,10 @@ def run():
for nb, info in all_nbs.items():

if "dependency" in info:
nbscuid.util.create_ploomber_nb_task(nb, info, cat_path, nb_path_root, output_dir, global_params, dag, dependency = info["dependency"])
cupid.util.create_ploomber_nb_task(nb, info, cat_path, nb_path_root, output_dir, global_params, dag, dependency = info["dependency"])

else:
nbscuid.util.create_ploomber_nb_task(nb, info, cat_path, nb_path_root, output_dir, global_params, dag)
cupid.util.create_ploomber_nb_task(nb, info, cat_path, nb_path_root, output_dir, global_params, dag)

#####################################################################
# Organizing scripts
Expand All @@ -109,14 +109,14 @@ def run():
for script, info in all_scripts.items():

if "dependency" in info:
nbscuid.util.create_ploomber_script_task(script, info, cat_path, nb_path_root, global_params, dag, dependency = info["dependency"])
cupid.util.create_ploomber_script_task(script, info, cat_path, nb_path_root, global_params, dag, dependency = info["dependency"])

else:
nbscuid.util.create_ploomber_script_task(script, info, cat_path, nb_path_root, global_params, dag)
cupid.util.create_ploomber_script_task(script, info, cat_path, nb_path_root, global_params, dag)

# Run the full DAG

dag.build()

return None

0 nbscuid/util.py → cupid/util.py
View file
Open in desktop
File renamed without changes.
4 changes: 2 additions & 2 deletions examples/adf-mom6/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ data_sources:
### to be created in
run_dir: .

# nb_path_root is the path to the folder that nbscuid will
# nb_path_root is the path to the folder that cupid will
### look for your template notebooks in. It doesn't have to
### be inside run_dir, or be specific to this project, as
### long as the notebooks are there
Expand All @@ -33,7 +33,7 @@ computation_config:
### specify a different environment than the default for any
### notebook in NOTEBOOK CONFIG

default_kernel_name: mom6-tools-nbscuid
default_kernel_name: cupid-analysis


############# NOTEBOOK CONFIG #############
Expand Down
2 changes: 1 addition & 1 deletion examples/adf-only/config-adf-only.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ data_sources:
### to be created in
run_dir: .

# nb_path_root is the path to the folder that nbscuid will
# nb_path_root is the path to the folder that cupid will
### look for your template notebooks in. It doesn't have to
### be inside run_dir, or be specific to this project, as
### long as the notebooks are there
Expand Down
6 changes: 3 additions & 3 deletions examples/nblibrary/config_f.cam6_3_119.FLTHIST_ne30.r328_gamma0.33_soae.001.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -199,7 +199,7 @@ diag_cam_baseline_climo:


#Location of baseline CAM climatologies:
cam_climo_loc: /glade/scratch/richling/adf-output/ADF-data/climos/${diag_cam_baseline_climo.cam_case_name}/${diag_cam_baseline_climo.yrs}/
cam_climo_loc: /glade/campaign/cesm/development/cross-wg/diagnostic_framework/ADF-data/climos/${diag_cam_baseline_climo.cam_case_name}/${diag_cam_baseline_climo.yrs}/

#model year when time series files should start:
#Note: Leaving this entry blank will make time series
Expand All @@ -226,7 +226,7 @@ diag_cam_baseline_climo:
cam_overwrite_ts: false

#Location where time series files are (or will be) stored:
cam_ts_loc: /glade/scratch/richling/adf-output/ADF-data/timeseries/${diag_cam_baseline_climo.cam_case_name}/${diag_cam_baseline_climo.yrs}/
cam_ts_loc: /glade/campaign/cesm/development/cross-wg/diagnostic_framework/ADF-data/timeseries/${diag_cam_baseline_climo.cam_case_name}/${diag_cam_baseline_climo.yrs}/

case_nickname: ${diag_cam_baseline_climo.cam_case_name}

Expand Down Expand Up @@ -368,4 +368,4 @@ tem_info:
overwrite_tem_case: false
overwrite_tem_base: false

#END OF FILE
#END OF FILE
16 changes: 9 additions & 7 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,16 @@ requires = ["setuptools>=61.0", "setuptools_scm[toml]>=6.2"]
build-backend = "setuptools.build_meta"

[tool.setuptools]
packages = ["nbscuid"]
packages = ["cupid"]


[project]
name = "nbscuid"
version = "0.0.10"
name = "cupid"
version = "0.0.1"
authors = [
{ name="Elena Romashkova", email="[email protected]" },
{ name="Lev Romashkov", email="[email protected]" },
{ name="Teagan King", email="[email protected]" },
{ name="Michael Levy", email="[email protected]" },
]
description = "Notebook-Based, Super CUstomizable Infrastructure for Diagnostics"
readme = "README.md"
Expand All @@ -36,10 +38,10 @@ dependencies = [
]

[project.urls]
repository = "https://github.com/rmshkv/nbscuid"
repository = "https://github.com/NCAR/CUPiD"
documentation = "https://nbscuid.readthedocs.io"
Copy link
Collaborator

@TeaganKing TeaganKing Jan 9, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be updated in #38 , but seems good to leave as is in this PR.



[project.scripts]
nbscuid-run = "nbscuid.run:run"
nbscuid-build = "nbscuid.build:build"
cupid-run = "cupid.run:run"
cupid-build = "cupid.build:build"