Add precipitable clouds in microwave all-sky assimilations (updated)

[jianjunj]

This is an update version of the PR #63 which resolves the issue NOAA-EMC/GDASApp#1333 .
The main update is that calculations of cloud fraction and cloud particle radius by the Thompson microphysics scheme are added.

Here is the summary of other added configurations in the original PR #63:
Add rain, snow, and graupel clouds for CRTM inputs.
Add a prior filter to zero clouds over non-water surfaces.
Add an option of clouds in the Hydrometeo check.
Add gross error check threshold for precipition impacted channels.
Add cold-air-outbreak filters.
Read cloud fraction from GeoVals.
Render list of not-bias corrected channels from JCB tables.
Add configurations for atms_n21.

It depends on two UFO PRs
https://github.com/JCSDA-internal/ufo/pull/3600
https://github.com/JCSDA-internal/ufo/pull/3573

It needs the develop branch in "global-workflow/sorc/gdas.cd/sorc/jcb" in order to rend non-bias corrected channels.

These UFO PRs are being revised. I'll inform you when it is ready to be merged.

[danholdaway]

Thanks @jianjunj !

[emilyhcliu]

Second thoughts .

1. We should remove the diagnostic flag variables since we have decided not to use the UKMO way for qcflags.

jcb-gdas/observations/atmosphere/atms_npp.yaml.j2

Lines 95 to 124 in d0868ff

	# Step 0-A: Create Diagnostic Flags
	- filter: Create Diagnostic Flags
	filter variables:
	- name: brightnessTemperature
	channels: *{{observation_from_jcb}}_simulated_channels
	flags:
	- name: ScanEdgeRemoval
	initial value: false
	force reinitialization: false
	- name: Thinning
	initial value: false
	force reinitialization: false
	- name: CLWRetrievalCheck
	initial value: false
	force reinitialization: false
	- name: WindowChannelExtremeResidual
	initial value: false
	force reinitialization: false
	- name: HydrometeorCheck
	initial value: false
	force reinitialization: false
	- name: GrossCheck
	initial value: false
	force reinitialization: false
	- name: InterChannelConsistency
	initial value: false
	force reinitialization: false
	- name: UseflagCheck
	initial value: false
	force reinitialization: false

And, the corresponding part of each action part of the filter should be changed, for example, from the following:

   actions:
    - name: set
      flag: WindowChannelExtremeResidual
    - name: reject

to

      action:
        name: reject

[emilyhcliu]

2. Add reduce obs space to two pre-filters (to mimic GSI)

• Domain check to remove observations from edge of the scan (Step 1)
• Gaussin check to thin data (Step 2)

# Step 1: Remove Observations from the Edge of the Scan
  - filter: Domain Check
    filter variables:
    - name: brightnessTemperature
      channels: 1-22
    where:
    - variable:
        name: MetaData/sensorScanPosition
      is_in: 7-90
    actions:
    - name: set
      flag: ScanEdgeRemoval
    - name: reject

  # Step 2: Data Thinning
  - filter: Gaussian Thinning
    horizontal_mesh: 145
    use_reduced_horizontal_grid: true
    distance_norm: geodesic
  #  round_horizontal_bin_count_to_nearest: true
  #  partition_longitude_bins_using_mesh: true
    actions:
    - name: set
      flag: Thinning
    - name: reject

Updated one:

# Step 1: Remove Observations from the Edge of the Scan
  - filter: Domain Check
    filter variables:
    - name: brightnessTemperature
      channels: 1-22
    where:
    - variable:
        name: MetaData/sensorScanPosition
      is_in: 7-90
    action:
      name: reject
      name: reduce obs space
   
  # Step 2: Data Thinning
  - filter: Gaussian Thinning
    horizontal_mesh: 145
    use_reduced_horizontal_grid: true
    distance_norm: geodesic
  #  round_horizontal_bin_count_to_nearest: true
  #  partition_longitude_bins_using_mesh: true
    action:
      name: reject
      name: reduce obs space

[jianjunj]

name: reduce obs space

@emilyhcliu It looks that we cannot add 'reduce obs space' in the configuration at the moment. It can only be in "obs filters". However, we need both prior or post filters, and "obs filters" cannot be added along with "obs prior fitlers" or "obs post filters".
Here is the error message:
Test "ufo/ObsFilters/testFilters" failed with unhandled eckit::Exception: Not implemented: Action 'reduce obs space' not implemented for prior or post filters yet @