Getting Started
DRAFT - These instructions are being updated for the v1.1.0 release and will not yet work as is. Check back soon.
The instructions on this page will enable you to build and run a simulation with initial conditions from August 29, 2019, during Hurricane Dorian. The simulation is configured for a C96 (~100km) resolution and will run for 2 days. The code base includes the UFS Weather Model, with pre- and post-processing, running in the Common Infrastructure for Modeling Earth (CIME) workflow.
More detailed instructions are available in the Medium-Range Weather Application Users Guide. If you have trouble with any of the commands in this section, it is a good idea to check the "Workflow Quick Start" section of the Users Guide for troubleshooting tips.
The instructions here assume you have defined $WORK, $HOME, and $SCRATCH directories on your system (and on some platforms, $USER).
To download the Medium-Range Weather Application, with externals pointing to its constituent components, go to your $HOME directory and enter:
$ git clone https://github.com/ufs-community/ufs-mrweather-app.git -b ufs-v1.1.0 my_ufs_sandbox
Then, to get the components:
$ cd my_ufs_sandbox
$ ./manage_externals/checkout_externals
See this link to determine the level of support for your platform and compiler. You will need to know if your platform is pre-configured, configurable, limited-test, or build-only.
There are a number of prerequisite libraries that must be installed before building and running this application. If you are running on a pre-configured platform, those libraries are already installed on your system.
If you are not running on a pre-configured platform, you will need to build the required libraries. The NCEPLIBS-external wiki page is the starting point for building the libraries.
Once you have the prerequisite libraries built, the workflow must be able to find them, via the environment variable NCEPLIBS_DIR. If you followed the instructions on the NCEPLIBS-external wiki page, the NCEPLIBS_DIR should be set to $WORK/NCEPLIBS-ufs-v1.1.0. On some systems (Stampede2, for example), you MUST source this file in your .bashrc or equivalent script in order for compute nodes to find it. For example, you would add the following to your .bashrc:
$ source $WORK/NCEPLIBS-ufs-v1.1.0/bin/setenv_nceplibs.sh
If you are on Hera or Cheyenne, there will be an input data directory with the sample data for this experiment all prepared. You need to set an environment variable to indicate this location to the App.
On Cheyenne:
$ export UFS_INPUT=$CESMDATAROOT
On Hera:
$ export UFS_INPUT=/scratch1/NCEPDEV/stmp2/CIME_UFS
If you are running on Hera or Cheyenne, you can skip ahead to the Set Environment Variables step. For all other platforms, you need to stage the raw initial conditions on disk, typically in a user scratch space, before you can run the experiment. First, you need to create top-level directory for staging the data for this case.
mkdir -p $SCRATCH/inputs/ufs_inputdata/icfiles/201908/20190829
Next, you need to retrieve the raw initial conditions file from a ftp site and link it to the name expected by the App.
cd $SCRATCH/inputs/ufs_inputdata/201908/20190829
wget -c https://ftp.emc.ncep.noaa.gov/EIB/UFS/inputdata/201908/20190829/gfs_4_20190829_0000_000.grb2
ln -s gfs_4_20190829_0000_000.grb2 atm.input.ic.grb2
Finally, you need to indicate where you staged the files.
$ export UFS_INPUT=$SCRATCH/inputs
You need to tell the App where to write the output files.
For example, on Cheyenne, use:
$ export UFS_SCRATCH=/glade/scratch/$USER
On Hera, use:
$ export UFS_SCRATCH=<my-project-dir>/$USER
On other platforms, similarly set $UFS_SCRATCH to a location on disk where you have permission to write.
There are three additional environment variables you need to set. You need to tell the App which compute project you will use.
$ export PROJECT=[Project ID]
The last two environment variables are:
$ export UFS_DRIVER=nems
$ export CIME_MODEL=ufs
Note that on Stampede2 you must also include the commands to set PROJECT, UFS_SCRATCH and UFS_INPUT environment variables in your .bashrc or equivalent script.
The next step is to set up a case. The create_newcase command creates a case directory containing the scripts and XML files to configure a case for the requested resolution, component set, and machine.
First change into the scripts directory:
$ cd $HOME/my_ufs_sandbox/cime/scripts
If you are running on a pre-configured or configurable platform, the following command will build your case. If not, you may need some additional arguments - see the Users Guide.
$ ./create_newcase --case DORIAN_C96_GFSv15p2 --compset GFSv15p2 --res C96 --workflow ufs-mrweather
The compset setting specifies the choice of atmospheric physics package (you can read more about the different options in the Users Guide). The workflow setting does not include post-processing, which translates the model outputs into the GRIB2 format on pressure levels. Model outputs for this example are in netCDF format on model levels.
Next you will create the scripts needed to run the model, and also the model namelist files:
$ cd DORIAN_C96_GFSv15p2
$ ./case.setup
Now you can build:
$ ./case.build
You'll need to modify the length of the run in env_run.xml. The default is 5 days. To set the run to 48 hours:
$ ./xmlchange STOP_OPTION=nhours,STOP_N=48
Since this is a fairly short, low-resolution run, the time requested can be reduced to 45 minutes.
$ ./xmlchange JOB_WALLCLOCK_TIME=00:45:00
$ ./xmlchange USER_REQUESTED_WALLTIME=00:45:00
The default start date is the start date for this Dorian example, so you don't need to adjust it.
Now you can run:
$ ./case.submit
Your outputs will be in the $SCRATCH/DORIAN_C96_GFSv15p2/run directory. After a successful run, the directory will look like this:
There are many netCDF visualization tools, such as Ncview, NCL, Ferret and Panoply.
For visually checking the results of your run, we have provided a NCL script that plots 2-m temperature (tmp2m) and total precipitation (tprcp) at 48 h.
You will need to load NCL first. Here are some examples of how to do this:
| System | Modules |
|---|---|
| Cheyenne | module load intel/18.0.5 ncl/6.6.2 |
| Stampede2 | module load ncl_ncarg/6.3.0 |
| Hera | module load ncl/6.5.0 |
Then get and run the script:
$ cd $SCRATCH/DORIAN_C96_GFSv15p2/run
$ wget https://raw.githubusercontent.com/wiki/ufs-community/ufs-mrweather-app/files/plot_ufs_sfcf.ncl
$ ncl plot_ufs_sfcf.ncl
The sample plots are below. They are consistent with the Hurricane Dorian initial conditions and tag ufs-v1.1.0. Your results may look different if you are using a different branch or tag. Your results will also look different just because you are running on a platform different from what we used to generate the plots.
