The topic definition is a Java package that allows identifying Envisat products by applying regular expressions-like file name matching rules. The central element of this package is the ontology definition file (cortex-index.owl) located in the META-INF folder inside the resources directory.
The topic is currently designed to support the following product types from the Envisat satellite mission:
- AATSR sensor
- ATS_LST_2P
- ASAR sensor
- ASA_WSM_1P
- ASA_IMM_1P
- ASA_APM_1P
- MERIS sensor
- MER_RR__2P
- MER_FRS_2P
- RA2 sensor
- RA2_GDR_2P
- RA2_MWS_2P
For decoding the binary datasets, which is necessary for extracting metadata and the footprint coordinates, product-specific XML schema definitions are contained in the (xsd) folder inside the resources directory. Where available, the metadata information is extracted from the XML metadata files provided within the product package.
The file name matching patterns and the XSD files were created based on the official product specification documents:
- ENVISAT-1 PRODUCTS SPECIFICATIONS, VOLUME 5: PRODUCT STRUCTURES
- ENVISAT-1 PRODUCTS SPECIFICATIONS, ANNEX A: PRODUCT DATA CONVENTIONS
- (A)ATSR Land Surface Temperature (LST) Product (UOL_LST_L2) Level 2 User Guide
- ENVISAT-1 PRODUCTS SPECIFICATIONS, VOLUME 7: AATSR PRODUCTS SPECIFICATIONS
- VOLUME 8: ASAR PRODUCTS SPECIFICATIONS
- ENVISAT-1 PRODUCTS SPECIFICATIONS, VOLUME 11: MERIS PRODUCTS SPECIFICATIONS
- ENVISAT-1 PRODUCTS SPECIFICATIONS, VOLUME 14: RA2 PRODUCTS SPECIFICATIONS LEVEL 2
Apache Maven is used for building the Java project. The Project Object Model XML file (pom.xml) specifies the overall package configuration.
The ATS_LST_2P product is provided in terms of netCDF files and corresponding md5 checksum files. It was agreed not the include the md5 files. It is possible, though, to combine these two individual files into one gzipped archive (tarball) and ingest the archive. The filename of the tarball needs to be exactly the same as for the nc/md5 files, with the obvious difference of having the file extension ".TGZ".
The following is an example illustrating the structure for the ATS_LST_2P product, after the previous step of tarball creation:
- ATS_LST_2PUUOL20020805_111312_000065272008_00194_02252_5972.TGZ
- ATS_LST_2PUUOL20020805_111312_000065272008_00194_02252_5972.nc
- ATS_LST_2PUUOL20020805_111312_000065272008_00194_02252_5972.md5
Note again that the file that will be ingested in DHuS is the ATS_LST_2PUUOL20020805_111312_000065272008_00194_02252_5972.TGZ in case both files shall be included. If the checksum file is to be ignored, the ATS_LST_2P product is directly the netCDF file:
- ATS_LST_2PUUOL20020805_111312_000065272008_00194_02252_5972.nc
No tarball is created in this case.
Metadata and footprints are created based on the information contained within the netCDF file.
The considered products from the Envisat ASAR sensor are provided as single files, i.e. one file per dataset:
- one mixed ASCII/BINARY data file having the extension .N1
Metadata and footprints are created based on the information contained within the N1 file.
An example for the ASA_WSM_1P product would be:
- ASA_WSM_1PNPDE20070304_113430_000001292056_00080_26186_0478.N1
Note that the file that will be ingested in DHuS is the N1 file itself, no tarball or compression is applied.
The same applies for the ASA_IMM_1P and ASA_APM_1P products.
Products from the Envisat MERIS sensor are provided in different formats. The currently available release online provides the products in the Envisat MERIS N1 format, i.e. one single file per dataset:
- one mixed ASCII/BINARY data file having the extension .N1
With the new release (as announced on 14 Jul 2020), the product format has been changed to the Sentinel-3 SAFE format (SEN3) based on a folder of netCDF files including an XML manifest file.
The present package is currently able to handle products in both the N1 and the SEN3 format, however, support for the N1 format will become obsolete.
The SEN3 products are folders containing netCDF files and an XML manifest file. In order for the DHuS ingest to work, the SEN3 folder needs to be zipped first.
The following is an example illustrating the structure for the MER_FRS_2P product as provided to EODC via NAS shipment, after the previous step of ZIP archive creation:
- ENV_ME_2_FRG____20020518T181911_20020518T182239_________________0208_006_070______DSI_R_NT____.SEN3.zip
- ENV_ME_2_FRG____20020518T181911_20020518T182239_________________0208_006_070______DSI_R_NT____.SEN3
- M01_rho_TOA.nc
- M01_rho_TOA.nc.md5
- M01_rho_top.nc
- ...
- xfdumanifest.xml
- ENV_ME_2_FRG____20020518T181911_20020518T182239_________________0208_006_070______DSI_R_NT____.SEN3
The file that will be ingested is ENV_ME_2_FRG____20020518T181911_20020518T182239_________________0208_006_070______DSI_R_NT____.SEN3.zip.
The exact delivery format of the MER_RR__2P is not yet clear as shipment to EODC has not been completed yet. An example for an MER_RR__2P from the ESA Envisat MERIS Online Dissemination Service is:
- EN1_MDSI_MER_RR__2P_20111224T204833_20111224T213212_051350_0043_20180901T153523_0100.ZIP
- ENV_ME_2_RRG____20111224T204833_20111224T213212_________________2618_110_043______DSI_R_NT____.SEN3
- chl_nn.nc
- chl_oc4me.nc
- ...
- xfdumanifest.xml
- ENV_ME_2_RRG____20111224T204833_20111224T213212_________________2618_110_043______DSI_R_NT____.SEN3
The file that will be ingested is EN1_MDSI_MER_RR__2P_20111224T204833_20111224T213212_051350_0043_20180901T153523_0100.ZIP. Note that the identifiers of the SEN3 folder and the ZIP file differ. Nonetheless, the addon is still capable of finding the SEN3 node inside the ZIP file.
In both cases, metadata information is extracted from the xfdumanifest.xml files.
The old product versions used the Envisat N1 data format. Metadata and footprints are created based on the information contained within the N1 file.
Examples for the MER_FRS_2P and the MER_RR__2P products would be:
- MER_FRS_2PPBCM20020618_101652_000000172007_00008_01565_0021.N1
- MER_RR__2PRBCM20020709_091801_000002222007_00308_01865_0002.N1
Note that the file that would be ingested in DHuS is the N1 file itself, no tarball or compression is applied.
The considered products from the RA2 sensor are provided in the netCDF format, i.e. one file per dataset:
- one netCDF file having the extension .nc
Examples for these products include:
- ENV_RA_2_GDR____20100915T134832_20100915T143849_20170826T150251_3017_093_0049____PAC_R_NT_003.nc
- ENV_RA_2_MWS____20041109T091841_20041109T100858_20170621T220348_3017_032_0015____PAC_R_NT_003.nc
No compression is applied in this case and the netCDF files are ingested directly into DHuS.
Metadata and footprints are created based on the information contained within the netCDF file.
The &envisat;productRA2_level2 class identifies the products from the RA2 sensor based on file name matching rules. The expected format is netCDF. This class must match the corresponding RA2 class name referenced by the metadata extractor in the Envisat addon (separate Java package).
The &envisat;productASAR_image class identifies Envisat ASAR Level-1 products. By using the SDF (Structured Data File) implementation for the the DRB (Data Request Broker) it is possible to break down the binary N1 files into a tree of nodes on the basis of XML schema definitions. This class must match the corresponding ASAR class name referenced by the metadata extractor in the Envisat addon (separate Java package).
The &envisat;productMERIS_level2_SEN3 class identifies the Envisat MERIS Level-2 SEN3 product folders. This class must match the corresponding MERIS SEN3 class name referenced by the metadata extractor in the Envisat addon (separate Java package).
This class is also used for the creation of quicklook images. The XML blocks img:descriptor and img:rendering contain settings and code for interacting with the corresponding Java routines contained inside this package. The code for generating the images is a copy of the one contained in official Sentinel-3 addon, with the difference that no conversion from radiance to reflectance values is performed, since MERIS Level-2 products already provide those.
This class identifies the old product versions of the MERIS Level-2 products, i.e. the binary N1 files. The SDF implementation is again used to decode the binary files based on the XML schema files in this package. Again, the names of the classes in the topic and the addon must match.
The &envisat;productATS_LST_2P class identifies the Envisat AATSR Level-2 product. No particular file extension is expected, because files might be ingested as archives or as pure netCDF files. This is the base class for the corresponding classes and its name must again match the one in the metadta extractor addon.
The &envisat;productATS_LST_2P_tgz is a subclass of &envisat;productATS_LST_2P and identifies the Envisat AATSR Level-2 product tarball.
The &envisat;productATS_LST_2P_netCDF is a subclass of &envisat;productATS_LST_2P and identifies the Envisat AATSR Level-2 product netCDF file.