Skip to content

shortwavetrough/CFSR

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

9 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

CFSR Request, Download, and Pre-Processing Program
--------------------------------------------------

	This program requests and downloads CFSR data, that is then 
	converted to GRIB (version 1) via convert_2_grib scripts. The 
	version 1 GRIB files are then ready to be pre-processed by 
	WPS. Unfortunately, there is a great deal of babysitting this 
	program while it runs, unlike downloading ERA-Interim data. 
	Currently, this is the most automated and quickest retrieval 
	version that I have developed to obtain CFSR data.

	It is strongly recommended to use version 1 GRIB files as 
	version 2 files are pre-processed significantly slower by WPS, 
	and WPS version 3.2 is not set up to work with version 2 GRIB.

	NCEP CFSR archived data has some corruption of data, so it is 
	strongly recommended the user checks for corruption before 
	pre-processing the data in WPS. This may simply be done by 
	checking the log files of data being output from 
	convert_2_grib scripts - corrupt data will not be converted, 
	and will result in an error being output.

	It is always recommended to check that all files were 
	successfully downloaded as the hardware systems tend to have 
	hiccups, resulting in failed downloads. It is good practice 
	to check both the number of files (ls -1 *.grb2 | wc -l) and 
	file sizes (for each f in *.grb2; do du -h $f ;done).

	There is a maximum of 10 requests/time. To view and delete 
	requests, log in to the archive at 
	rda.ucar.edu/datasets/ds093.0 and navigate to your Dashboard. 
	After 10 requests have been submitted, the user will need to 
	download the data. Further requests may then be made. The 
	batchreq.sh and batchdl.sh scripts allow data to be requested 
	and downloaded in sets not exceeding 10/time by adjusting the 
	"FIRSTPASS" flag.

How to use this program:

scripts/
        1.  batchreq.sh - Requests the data that will be downloaded. 
			  Before running this script, add your UCAR 
			  password to the environment flag "pswd", and 
			  edit the script to fit your data and 
			  workspace.
	2.  After batchreq.sh is complete, the user will need to 
	    navigate to their Dashboard and obtain the perl download 
	    scripts for each request. Save the individual perl 
	    scripts as "get_file.pl" to the respective variable 
	    directories. The environment variable "pswd" should be 
	    set to your UCAR password.
	3.  batchdl.sh - Downloads the data. Before running this 
			 script, add your UCAR password to the 
			 "RDAPSWD" variable, and edit the script to
			 fit your data and workspace.
	4.  After the downloads have completed, navigate to your 
	    Dashboad and delete the data. The user will then be ready 
	    to repeat steps (1) - (4) to obtain additional data.
	5.  convert_2_grib.sh - Converts the version 2 GRIB data to 
				version 1 GRIB. Before running this 
				script, edit the script to fit your 
				data and workspace.

WPS_scripts/
	The namelists are from WPS version 3.2, and it is assumed the 
	user is using this version. The shell scripts should be one 
	directory up from the WPS/ directory. The namelist files, 
	pbs files, and Vtable file should reside in the WPS/ 
	directory. The geo_em.d01.nc NetCDF file should reside in 
	WPS/geog/ directory.
	1.  runungrib.csh - Runs ungrib for a period of time specified 
			    by the user. Before running this script, 
			    edit the script to fit your workspace.
	2.  runmetgrid.csh - Runs metgrid for a period of time 
			     specified by the user. Before running 
			     this script, edit the script to fit your 
			     workspace.
	3.  ungrib.pbs - Defines the commands used for Ungrib. Before 
			 running runungrib.csh, edit this script by 
			 adding in the queue to use, job number, and 
			 memory and cpu usage.
	4.  metgrid.pbs - Defines the commands used for Metgrid. 
			  Before running runmetgrid.csh, edit this 
			  script by adding in the queue to use, job 
			  number, and memory and cpu usage.
	5.  namelist.wps.[SFC,PL] - Namelist information Ungrib will 
				    use for surface and pressure level 
				    variables.
	6.  namelist.wps.STD - Namelist information Metgrid will use.
	7.  Vtable - Variables to unpack from CFSR GRIB (version 1).
	8.  geo_em.d01.nc - Output from Geogrid defining the model 
			    domain.

noleap/
	1.  mod_metem.ncl - At this time RASM does not use leap days, 
			    so this script replaces the leap day data 
			    with data from March 01. This allows for 
			    correct calculation of tendencies 
			    performed by real.exe. This script must be 
			    run for all leap years, and before 
			    running real.exe.
	2.  mod_wrf[bdy,fdda].ncl - These scripts remove the leap day 
				    from the lateral boundary and 
				    spectral nudging files. Run after 
				    real.exe completes.

real_scripts/
	1.  runreal.CFSR.csh - Sets up the namelist and PBS script 
			       needed to run real.exe. This script 
			       assumes the directory containing WRF is 
			       named "WRFV3_single." Before running, 
			       edit this script to fit your data and 
			       workspace.
	2.  real.CFSR.pbs - Defines the commands used for real.exe. 
			    Before running runreal.CFSR.csh, edit this 
			    script to fit your workspace.
	3.  namelist.input.racm_input - Namelist information real.exe 
					will use.

conf_scripts/
	WRF must be compiled before WPS. The configuration scripts 
	are architecture dependent and will likely not work on other 
	machines. The JasPer, PNG, and Zlib libraries are only needed 
	for GRIB version 2 support. It is strongly recommended to not 
	use GRIB version 2 data! Native WPS 3.2.x does not support 
	version 2 GRIB files.
	1.  configure.wrf - WRF configuration script. The user may 
			    need to edit the architecture settings to 
			    the respective compiler details of the 
			    system WRF will be compiled on.
	2.  configure.wps - WPS configuration script. The user may 
			    need to edit "COMPRESSION_LIBS", 
			    "COMPRESSION_INC", "NCARG_LIBS", 
			    "WRF_DIR", and architecture settings to 
			    the respective paths and compiler details 
			    of the system WPS will be compiled on.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages