Skip to content

Commit

Permalink
Updated for new release - 1.1.5; CMOR3.4.0
Browse files Browse the repository at this point in the history
  • Loading branch information
durack1 committed Jan 24, 2019
1 parent 8987922 commit a3e224f
Show file tree
Hide file tree
Showing 17 changed files with 301 additions and 159 deletions.
5 changes: 3 additions & 2 deletions CMOR/drive_input4MIPs_bcs.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"institution_id": "PCMDI",
"source_id": "PCMDI-AMIP-1-1-4",
"source": "PCMDI-AMIP 1.1.4: Merged SST based on UK MetOffice HadISST and NCEP OI2",
"source_id": "PCMDI-AMIP-1-1-5",
"source": "PCMDI-AMIP 1.1.5: Merged SST based on UK MetOffice HadISST and NCEP OI2",
"contact ": "PCMDI ([email protected])",
"further_info_url": "https://pcmdi.llnl.gov/mips/amip",

Expand All @@ -15,6 +15,7 @@
"data_usage_tips": "The mid-month data should be linearly interpolated in time and then clipped for use as boundary conditions to drive AMIP simulations as described at: https://pcmdi.llnl.gov/mips/amip",

"outpath": "/p/user_pub/work",
"#outpath": "/work/durack1/Shared/150219_AMIPForcingData",
"_history_template": "%s; CMOR rewrote data to be consistent with <activity_id>, CMIP6, and <Conventions> standards",
"output_path_template": "<activity_id><mip_era><target_mip><institution_id><source_id><realm><frequency><variable_id><grid_label><version>",
"output_file_template": "<variable_id><activity_id><dataset_category><target_mip><source_id><grid_label>",
Expand Down
5 changes: 3 additions & 2 deletions CMOR/drive_input4MIPs_obs.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"institution_id": "PCMDI",
"source_id": "PCMDI-AMIP-1-1-4",
"source": "PCMDI-AMIP 1.1.4: Merged SST based on UK MetOffice HadISST and NCEP OI2",
"source_id": "PCMDI-AMIP-1-1-5",
"source": "PCMDI-AMIP 1.1.5: Merged SST based on UK MetOffice HadISST and NCEP OI2",
"contact ": "PCMDI ([email protected])",
"further_info_url": "https://pcmdi.llnl.gov/mips/amip",

Expand All @@ -13,6 +13,7 @@
"region": "global_ocean",

"outpath": "/p/user_pub/work",
"#outpath": "/work/durack1/Shared/150219_AMIPForcingData",
"_history_template": "%s; CMOR rewrote data to be consistent with <activity_id>, CMIP6, and <Conventions> standards",
"output_path_template": "<activity_id><mip_era><target_mip><institution_id><source_id><realm><frequency><variable_id><grid_label><version>",
"output_file_template": "<variable_id><activity_id><dataset_category><target_mip><source_id><grid_label>",
Expand Down
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
# amipbcs - AMIP dataset prepared for input4MIPs
Code to generate boundary condition data for the AMIP (Atmospheric Model Intercomparison Project) experiment as part of CMIP6 and the input4MIPs project

Generated data can be obtained from the [PCMDI ESGF data portal](https://esgf-node.llnl.gov/search/input4mips/?dataset_version_number=1.1.4)
Generated data can be obtained from the [PCMDI ESGF data portal](https://esgf-node.llnl.gov/search/input4mips/?source_version=1.1.5)

The code depends upon the following packages:
- [**CDAT 8.0 with cd77**](https://github.com/UV-CDAT/uvcdat) (Available through [conda](https://anaconda.org/uvcdat/uvcdat/files))
- [**CMOR 3.3.2+**](https://github.com/PCMDI/cmor) (Available through [conda](https://anaconda.org/PCMDI/cmor/files))
- [**CDAT 8.0+ with cd77**](https://github.com/CDAT/cdat) (Available through [conda](https://anaconda.org/CDAT/repo))
- [**CMOR 3.4.0+**](https://github.com/PCMDI/cmor) (Available through [conda](https://anaconda.org/conda-forge/cmor/files))
- [**gfortran 4.8.5+**](https://gcc.gnu.org/wiki/GFortran)

The optional libraries of CDAT 8.0 are required:
The optional libraries of CDAT 8.1+ are required (these are bundled with cd77):
- [EzGet](https://github.com/UV-CDAT/EzGet)
- [lats](https://github.com/UV-CDAT/lats)
- [libdrs](https://github.com/UV-CDAT/libdrs)
Expand Down
214 changes: 141 additions & 73 deletions README.txt
Original file line number Diff line number Diff line change
@@ -1,109 +1,177 @@
Fri 27 Apr 2018 01:06:22 PM PDT
Thu 24 Jan 2019 02:30:52 PM PST

The following documents the steps to update the boundary conditions:

1. Install or update Anaconda
> Visit https://www.continuum.io/downloads and follow the prompts for your OS
> durack1@oceanonly:[150219_AMIPForcingData]:[6913]> conda update --prefix /export/durack1/anaconda2 anaconda
> Visit https://www.continuum.io/downloads and follow the prompts for your OS - 2018.12 is latest

2a. Install CDAT, cd77 (ezget/lats/drs support), CMOR3, NCL and NCO
durack1@oceanonly:[150219_AMIPForcingData]:[6918]> conda create -n cdat80cmor332nclnco -c conda-forge -c cdat cdat 'python<=3'
# cdat: 8.0-0 cdat; cdat_info: 8.0-py27_0 conda-forge; cdms2: 3.0-py27_5 conda-forge; cdutil: 8.0-py27_1 conda-forge
durack1@oceanonly:[150219_AMIPForcingData]:[6919]> conda activate cdat80cmor332nclnco
(cdat80cmor332nclnco) durack1@oceanonly:[150219_AMIPForcingData]:[6920]>
[durack1@oceanonly ~]$ conda update --prefix /export/durack1/anaconda2 anaconda
Solving environment: done

# All requested packages already installed.


2a. Install cdms2, cdutil, cd77 (ezget/lats/drs support), CMOR3, NCL, NCO and flex (for wrapit77)
[durack1@oceanonly ~]$ conda create -n cdmsNcd77Ncmor340nclncoflex -c cdat/label/nightly -c conda-forge cdms2 cdutil cd77 cmor ncl nco flex 'python<=3'
# cdms2: 3.1.0.2018.11.29.19.18.g5a83dd4-py27h481b005_0 cdat/label/nightly; cdat_info: 8.0.2018.11.28.19.22.gb6530fa-py_0 cdat/label/nightly;
# cdutil: 8.0.2018.11.12.19.20.g3bd6383-py_0 cdat/label/nightly; cd77: 1.1-py27_0 cdat/label/nightly; cmor: 3.4.0-py27ha570855_0 conda-forge;
# ncl: 6.5.0-blas_openblash04324b8_3 conda-forge [blas_openblas]; nco: 4.7.8-he007dc3_0 conda-forge; flex: 2.6.4-hf484d3e_1004 conda-forge

Installing flex solves issues
***
This solves issues with wrapit77

Hit issues with wrapit77 -

WRAPIT Version: 120209
OPERATING SYSTEM: Linux
nbits = 64
FORTRAN COMPILER (f90c): gfortran
FORTRAN COMPILER OPTIONS (fopts): -m64 -fPIC -v -c -fno-second-underscore
/export/durack1/anaconda2/envs/cdat80cmor333nclnco/bin/wrapit77: error while loading shared libraries: libfl.so.2: cannot open shared object file: No such file or directory
FATAL ERROR: wrapit77 failed
***

[durack1@oceanonly ~]$ bash
ANACONDA:/usr/local/anaconda2

You are logged into a Linux Machine...(Version 2.6.32-754.3.5.el6.x86_64)
Host => oceanonly.llnl.gov

Hardware: x86_64....
Using Emacs Bindings...
11:23:03 up 111 days, 23:55, 10 users, load average: 0.28, 0.61, 0.60

(psst... may want to install "fortune")

/export/durack1/anaconda2/etc/profile.d/conda.sh
bash: conda: command not found
/export/durack1/anaconda2/etc/profile.d/conda.sh
durack1@oceanonly:[~]:[13752]> source activate cdmsNcd77Ncmor340nclncoflex
(cdmsNcd77Ncmor340nclncoflex) durack1@oceanonly:[~]:[13753]>

2b. Validate hdf libraries are correctly installed
(cdat80cmor332nclnco) durack1@oceanonly:[150219_AMIPForcingData]:[6920]> ls /export/durack1/anaconda2/envs/cdat80cmor332nclnco/lib/libmfhdf.*
/export/durack1/anaconda2/envs/cdat80cmor332nclnco/lib/libmfhdf.a
/export/durack1/anaconda2/envs/cdat80cmor332nclnco/lib/libmfhdf.la
/export/durack1/anaconda2/envs/cdat80cmor332nclnco/lib/libmfhdf.so
/export/durack1/anaconda2/envs/cdat80cmor332nclnco/lib/libmfhdf.so.0
/export/durack1/anaconda2/envs/cdat80cmor332nclnco/lib/libmfhdf.so.0.0.0

2c. Install cd77, cmor, ncl, nco
(cdat80cmor332nclnco) durack1@oceanonly:[150219_AMIPForcingData]:[6921]> conda install -c uvcdat cd77
# cd77: 1.0.0-py27_2 uvcdat
(cdat80cmor332nclnco) durack1@oceanonly:[150219_AMIPForcingData]:[6918]> conda install -c pcmdi cmor
# cmor: 3.3.2.npy1.13-py27hb8bc26d_0 pcmdi
(cdat80cmor332nclnco) durack1@oceanonly:[150219_AMIPForcingData]:[6917]> conda install -c conda-forge ncl
# ncl: 6.4.0-blas_openblas_9 conda-forge [blas_openblas] AND OTHER DEPENDENCIES
(cdat80cmor332nclnco) durack1@oceanonly:[150219_AMIPForcingData]:[6918]> conda install -c conda-forge nco
# nco: 4.7.4-0 conda-forge

3. Copy previous directory contents to new directory
(cdat80cmor332nclnco) durack1@oceanonly:[150219_AMIPForcingData]:[6909]> cp -R SST_1-1-3 SST_1-1-4

4. Purge generated files and set perms to execute
(cdat80cmor332nclnco) durack1@oceanonly:[150219_AMIPForcingData]:[6912]> cd SST_1-1-4/
/work/durack1/Shared/150219_AMIPForcingData/SST_1-1-4
(cdat80cmor332nclnco) durack1@oceanonly:[SST_1-1-4]:[6913]> rm -f 171023*.* MODEL.*.nc
(cdat80cmor332nclnco) durack1@oceanonly:[SST_1-1-4]:[6878]> chmod 755 *.*
(cdmsNcd77Ncmor340nclncoflex) durack1@oceanonly:[~]:[13753]> ls -al /export/durack1/anaconda2/envs/cdmsNcd77Ncmor340nclncoflex/lib/libmfhdf.*
-rw-rw-r-- 3 durack1 climate 295K Oct 18 04:20 /export/durack1/anaconda2/envs/cdmsNcd77Ncmor340nclncoflex/lib/libmfhdf.a
lrwxrwxrwx 1 durack1 climate 17 Jan 17 11:23 /export/durack1/anaconda2/envs/cdmsNcd77Ncmor340nclncoflex/lib/libmfhdf.so -> libmfhdf.so.0.0.0
lrwxrwxrwx 1 durack1 climate 17 Jan 17 11:23 /export/durack1/anaconda2/envs/cdmsNcd77Ncmor340nclncoflex/lib/libmfhdf.so.0 -> libmfhdf.so.0.0.0
-rwxrwxr-x 3 durack1 climate 187K Oct 18 04:20 /export/durack1/anaconda2/envs/cdmsNcd77Ncmor340nclncoflex/lib/libmfhdf.so.0.0.0


3. Copy previous directory contents to new directory (NOTE THIS WAS COMPLETED IN OCTOBER, so just replicating what was done back then along with edited scripts)
(cdmsNcd77Ncmor340nclncoflex) durack1@oceanonly:[150219_AMIPForcingData]:[9632]> cp -R SST_1-1-4 SST_1-1-5b
(cdmsNcd77Ncmor340nclncoflex) durack1@oceanonly:[150219_AMIPForcingData]:[9633]> cd SST_1-1-5b
/work/durack1/Shared/150219_AMIPForcingData/SST_1-1-5b


4. Purge previous files and edit contents to reflect latest downloads
(cdmsNcd77Ncmor340nclncoflex) durack1@oceanonly:[SST_1-1-5b]:[9634]> rm -f 180416* ; # Cleanup old files
(cdmsNcd77Ncmor340nclncoflex) durack1@oceanonly:[SST_1-1-5b]:[9638]> rm -f MODEL.*.nc ; # Cleanup old files
(cdmsNcd77Ncmor340nclncoflex) durack1@oceanonly:[SST_1-1-5b]:[9634]> rsync ../SST_1-1-5/download.sh . ; # Copy files from previous v1.1.5 update
(cdmsNcd77Ncmor340nclncoflex) durack1@oceanonly:[SST_1-1-5b]:[9634]> rsync ../SST_1-1-5/SSTICE.Update.unf.csh . ; # Copy files from previous v1.1.5 update


5. Edit script files (there is a need to update PATH info to include the binaries installed into the conda env)
> download.sh & SSTICE.Update.unf.csh
# Make sure to set environment variables NCARG_ROOT and PATH in SSTICE.Update.unf.csh to /export/durack1/anaconda2/envs/cdat80cmor332nclnco/bin:${PATH}
# Make sure to set environment variables NCARG_ROOT and PATH in SSTICE.Update.unf.csh to /export/durack1/anaconda2/envs/cdmsNcd77Ncmor340nclncoflex/bin:${PATH}
# Correct all indexing to reflect download updates


6. Run
(cdat80cmor332nclnco) durack1@oceanonly:[SST_1-1-4]:[6880]> download.sh
(cdmsNcd77Ncmor340nclncoflex) durack1@oceanonly:[SST_1-1-5b]:[9642]> download.sh


7. Edit fortran code
7. Jump back up to master directory; Edit fortran code
(cdmsNcd77Ncmor340nclncoflex) durack1@oceanonly:[SST_1-1-5b]:[9642]> cd ..
> mkhurrell1.f
> increment version number v1.1.3 -> v1.1.4
> increment version number v1.1.4 -> v1.1.5 (for this test use latest data SST_1-1-5b)
> Update start/end dates and array sizes (Should be ok for a while, this update extended to 160 [2030-1870 = 160])

8. Compile and set environment variables
(cdat80cmor332nclnco) durack1@oceanonly:[SST_1-1-4]:[6880]> which cd77
/export/durack1/anaconda2/envs/cdat80cmor332nclnco/bin/cd77
(cdat80cmor332nclnco) durack1@oceanonly:[SST_1-1-4]:[6864]> gfortran --version
GNU Fortran (GCC) 4.8.5
...
(cdat80cmor332nclnco) durack1@oceanonly:[SST_1-1-4]:[6860]> ls ../src/
drsdef.h ketgrib.parms lats.inc
(cdat80cmor332nclnco) durack1@oceanonly:[150219_AMIPForcingData]:[6832]> cd ..
(cdat80cmor332nclnco) durack1@oceanonly:[150219_AMIPForcingData]:[6832]> ls -al mkhurrell1.f
-rwxr-xr-x 1 durack1 climate 214K Apr 16 17:06 mkhurrell1.f
(cdat80cmor332nclnco) durack1@oceanonly:[150219_AMIPForcingData]:[6831]> cd77 -ezget -lats -cdms -fcray-pointer mkhurrell1.f -o mkhurrell1
(cdat80cmor332nclnco) durack1@oceanonly:[150219_AMIPForcingData]:[6834]> chmod 755 mkhurrell1

8. Compile
(cdmsNcd77Ncmor340nclncoflex) durack1@oceanonly:[150219_AMIPForcingData]:[9643]> cd77 -ezget -lats -cdms -fcray-pointer mkhurrell1.f -o mkhurrell1
f951: Warning: Nonexistent include directory '/export/durack1/anaconda2/envs/cdmsNcd77Ncmor340nclncoflex/include/cdms' [-Wmissing-include-dirs]
f951: Warning: Nonexistent include directory '/export/durack1/anaconda2/envs/cdmsNcd77Ncmor340nclncoflex/lib/libffi-3.1/include' [-Wmissing-include-dirs]
f951: Warning: Nonexistent include directory '/usr/X11R6/include' [-Wmissing-include-dirs]
(cdmsNcd77Ncmor340nclncoflex) durack1@oceanonly:[150219_AMIPForcingData]:[9643]> ls -al src
total 44K
drwxr-xr-x 2 durack1 climate 4.0K Apr 19 2017 .
drwxr-xr-x 30 durack1 climate 4.0K Jan 18 2019 ..
-rwxr-xr-x 1 durack1 climate 8.1K Apr 19 2017 drsdef.h
-rwxr-xr-x 1 durack1 climate 14K Jun 16 2015 ketgrib.parms
-rwxr-xr-x 1 durack1 climate 5.5K Apr 19 2017 lats.inc


9. Run
# Create/prepare output directory
(cdat80cmor332nclnco) durack1@oceanonly:[150219_AMIPForcingData]:[6825]> mkdir 360x180_v1.1.4
(cdat80cmor332nclnco) durack1@oceanonly:[150219_AMIPForcingData]:[6826]> ./mkhurrell1
(cdmsNcd77Ncmor340nclncoflex) durack1@oceanonly:[150219_AMIPForcingData]:[9643]> mkdir 360x180_v1.1.5
(cdmsNcd77Ncmor340nclncoflex) durack1@oceanonly:[150219_AMIPForcingData]:[9639]> mkhurrell1
*****
# If errors are hit, add LD_LIBRARY_PATH to env
mkhurrell1: error while loading shared libraries: libmfhdf.so.0: cannot open shared object file: No such file or directory
# KLUDGE Add libmfhdf.so.0 to path
(cdat80cmor332nclnco) durack1@oceanonly:[150219_AMIPForcingData]:[5087]> export LD_LIBRARY_PATH=/export/durack1/anaconda2/envs/cdat80cmor332nclnco/lib
*****

10. Edit sanitize script

10a. Edit sanitize script
> sanitize.py ; # Be aware of complete vs partial years with calendar creation

11. Update (or clone) the input4MIPs-cmor-table repo

10b. Register new PCMDI-AMIP-1-1-5 in input4mips-cmor-tables
> Update all details to reflect latest version

10c. Update to latest input4mips-cmor-tables
[durack1@oceanonly input4MIPs-cmor-tables]$ cd
[durack1@oceanonly input4MIPs-cmor-tables]$ cd git/input4MIPs-cmor-tables/
[durack1@oceanonly input4MIPs-cmor-tables]$ git fetch --all -p
[durack1@oceanonly input4MIPs-cmor-tables]$ git checkout master
[durack1@oceanonly input4MIPs-cmor-tables]$ git pull
# Check read perms on input4mips-cmor-tables/Tables subdir


12. Make any required tweaks to CMOR json input
(cdat80cmor332nclnco) durack1@oceanonly:[150219_AMIPForcingData]:[5552]> ls CMOR
drive_input4MIPs_bcs.json drive_input4MIPs_obs.json input4MIPs-cmor-tables
10d. Edit CMOR input jsons
(cdmsNcd77Ncmor340nclncoflex) durack1@oceanonly:[150219_AMIPForcingData]:[9639]> ls -al CMOR
total 16K
drwxr-xr-x 2 durack1 climate 4.0K Apr 27 2018 .
drwxr-xr-x 30 durack1 climate 4.0K Jan 18 12:25 ..
-rwxr-xr-x 1 durack1 climate 2.7K Jan 4 13:54 drive_input4MIPs_bcs.json
-rwxr-xr-x 1 durack1 climate 2.5K Jan 4 13:55 drive_input4MIPs_obs.json
lrwxrwxrwx 1 durack1 climate 42 Jun 2 2016 input4MIPs-cmor-tables -> /export/durack1/git/input4MIPs-cmor-tables
# Migrate as much content from CMOR input jsons to source_id registration

13. Load cmor environment and run (this now writes direct to destination for publication)
(cdat80cmor332nclnco) durack1@oceanonly:[150219_AMIPForcingData]:[4937]> python sanitize.py

14. Validate data against previously published version - edit validation script
11. Cleanup previous v1.1.5 versions
(cdmsNcd77Ncmor340nclncoflex) durack1@oceanonly:[150219_AMIPForcingData]:[9632]> rm -rf SST_1-1-5 SST_1-1-5a
(cdmsNcd77Ncmor340nclncoflex) durack1@oceanonly:[150219_AMIPForcingData]:[9634]> rm -rf 360x180_v1.1.5_san 360x180_v1.1.5_old 360x180_v1.1.5_old2
(cdmsNcd77Ncmor340nclncoflex) durack1@oceanonly:[150219_AMIPForcingData]:[9635]> rm -rf input4MIPs/CMIP6/CMIP/PCMDI/PCMDI-AMIP-1-1-5


12. Edit and run sanitize.py (Write to local directory for testing)
(cdmsNcd77Ncmor340nclncoflex) durack1@oceanonly:[150219_AMIPForcingData]:[9635]> python sanitize.py
# Edits required for new version, dates etc
# Note test before writing direct to publication destination


12a. Validate data against previously published version - edit validation script
> make_newVsOldDiffs.py ; # Be wary of any unit changes
# Source cdat80 environment and run movie generation script
(cdat80py2) durack1@oceanonly:[150219_AMIPForcingData]:[11026]> python make_newVsOldDiffs.py > 190118_1322_make_newVsOldDiffs.txt

15. Source cdat80 environment and run movie generation script
(cdat80) durack1@oceanonly:[150219_AMIPForcingData]:[8079]> python make_newVsOldDiffs.py > 180427_1217_make_newVsOldDiffs.txt

16. Validate output movies
12b. Validate output movies
# Pay particular attention to end points (2016-)

18. Publish data to ESGF
> Will need to involve Sasha/Tony for this step

19. Update github repo
12c. Edit path and rerun sanitize.py (Now writing directly to destination for publication)
(cdmsNcd77Ncmor340nclncoflex) durack1@oceanonly:[150219_AMIPForcingData]:[9635]> python sanitize.py


13. Publish data to ESGF
> Will need to involve Sasha/Jiwoo for this step


14. Update github repo
> cd /export/durack1/git/amipbcs
# Update all source files
> rsync -vrut /work/durack1/Shared/150219_AMIPForcingData/*.f .
Expand All @@ -115,16 +183,16 @@ drive_input4MIPs_bcs.json drive_input4MIPs_obs.json input4MIPs-cmor-tables
> rsync -vrut /work/durack1/Shared/150219_AMIPForcingData/CMOR/ CMOR/
> chmod 644 CMOR/*.json ; # Update file permissions
# Update data contributed (*.nc files are not hosted in git - .gitignore)
> #rm -rf SST_1-1-3 ; # Purge local dir (if offical release replaces it)
> rsync -vrut /work/durack1/Shared/150219_AMIPForcingData/SST_1-1-4 . ; # Copy remote dir
> chmod 755 SST_1-1-4 ; # Update file permissions
> chmod 755 SST_1-1-4/*.csh SST_1-1-4/*.dat SST_1-1-4/*.f SST_1-1-4/*.ncl SST_1-1-4/*.sh
> chmod 644 SST_1-1-4/*.bz2 SST_1-1-4/*.nc SST_1-1-4/*.txt
> #rm -rf SST_1-1-4 ; # Purge local dir (if offical release replaces it)
> rsync -vrut /work/durack1/Shared/150219_AMIPForcingData/SST_1-1-5b . ; # Copy remote dir
> chmod 755 SST_1-1-5b ; # Update file permissions
> chmod 755 SST_1-1-5b/*.csh SST_1-1-5b/*.dat SST_1-1-5b/*.f SST_1-1-5b/*.ncl SST_1-1-5b/*.sh
> chmod 644 SST_1-1-5b/*.bz2 SST_1-1-5b/*.nc SST_1-1-5b/*.txt
# Update this file once everything is updated
> rsync -vrut /work/durack1/Shared/150219_AMIPForcingData/README.txt .
# Update README.md
# Add new dir to repo
> git add SST_1-1-4
> git add SST_1-1-5b
# Commit version
> git commit -am 'Updated for new release - 1.1.4; CMOR3.3.2'
> git commit -am 'Updated for new release - 1.1.5; CMOR3.4.0'
> git push
Binary file removed SST_1-1-4/180416.tar.bz2
Binary file not shown.
Loading

0 comments on commit a3e224f

Please sign in to comment.