Home

Welcome to the monetio wiki!

Adding new data sources

Requirements for Observations

Requirements for new observations are to be described here for now.

• Point observations should have a pandas DataFrame, e.g. aqs.df

• Ways to read and retrieve if possible. CRN, AIRNOW, AQS as examples retrieve data if needed. IMPROVE can read data downloaded manually.
  method retrieve
  determine filename and FTP address from inputs
  if download==False then
  return ftp address and filename.
  if download==True then
  check to see if file already exists.
  if it doesn't, then retrieve the file from ftp site (or other location)
  return filename
  method load
  load data into pandas dataframe.
  input may be an ftp address or a filename.
  If an ftp address then data should be read directly into dataframe not
  downloaded first.
  method str
  return string with pertinent information about the observations.
  may include default url for ftp retrievals.

• Units are required for all measurements in a separate field. The units need to be converted to:

  • Kelvin
  • ppbv
  • Dobson
  • ug/m3
  • w/m2
  • m/s

• Observations should have a column with 'latitude' and 'longitude' if a surface measurement with a pandas DataFrame.

• Point measurements need a 'siteid' data column identifying the measurement site. This could be a float, integer, or string type.

• Observations should have a 'time' column which is the GMT date/time.

• All observations should have an objtype attribute which describes the dataset. For instance, airnow has self.objtype = 'AirNow'

Requirements for Models and 2-D Observations

Requirements for new multidimensional objects

• All datasests should be store as a xarray Dataset or opened as an xarray DataArray.

• If the data can be retrieved via opendap or ftp a method should be created to do so.

• All model data should be opened with a method called dataobj.open_dataset(....) which returns the xarray object

• Units are required for all fields and should be stored as an attribute. The units need to be converted to:

  • Kelvin
  • ppbv
  • Dobson
  • ug/m3
  • w/m2
  • m/s
  • etc.....

• All datasets should include latitude and longitude as coordinates in the xarray Dataset with dimensions (y, x)

• Dimensions should have the names x, y, z, and time where time is the GMT date/time