From 2d35b0c2d581c9d854f6c94a03f906805055ca58 Mon Sep 17 00:00:00 2001 From: Stefaan Lippens Date: Mon, 5 Feb 2024 16:55:58 +0100 Subject: [PATCH] Move WP6 (workflow) specifics to root readme --- README.md | 48 +++++++++++++++++++ .../tests/workflows/README.md | 35 ++------------ 2 files changed, 51 insertions(+), 32 deletions(-) diff --git a/README.md b/README.md index d45ebcb..f5e722f 100644 --- a/README.md +++ b/README.md @@ -270,6 +270,54 @@ which is driven by the `OPENEO_AUTH_METHOD` environment variable: for instructions on how to complete the authentication flow. +## Test module specifics + +Some test modules have specific considerations and options to be aware of. + +### WP6 Full process graph execution and validation + +These tests are designed to run using synchronous openEO process calls +and a Sentinel-2(-like) collection. + +The S2 collection to use must be specified through the `--s2-collection` option, +which supports two forms: + +- a normal openEO collection name (e.g. `--s2-collection=SENTINEL2_L2A`) + which will be loaded through the standard openEO `load_collection` process. +- a STAC URL (typically starting with `https://`), + which will be loaded with `load_stac`. + + The following two example STAC collections are provided in the context of this test suite project. + Each of these exists to cater to subtle differences between some back-end implementations + regarding temporal and band dimension naming and how that is handled in `load_stac`. + - https://stac.eurac.edu/collections/SENTINEL2_L2A_SAMPLE: + defines `"t"` as temporal dimension name, and `"bands"` as bands dimension name. + Recommended to be used with VITO/CDSE openEO backends. + - https://stac.eurac.edu/collections/SENTINEL2_L2A_SAMPLE_2: + uses `"time"` as temporal dimension name, and `"band"` as bands dimension name. + Recommended to be used with EURAC and EODC openEO backends. + +If the back-end does not support `load_stac`, a collection name must be used. + +#### Usage examples: + +```bash +# Compact +pytest src/openeo_test_suite/tests/workflows \ + -U openeo.dataspace.copernicus.eu \ + --s2-collection=SENTINEL2_L2A + +# With full back-end URL, a STAC collection to use instead of a predefined openEO collection +# and a list of process levels to test against +pytest src/openeo_test_suite/tests/workflows \ + --openeo-backend-url=https://openeo.dataspace.copernicus.eu/openeo/1.2 \ + --s2-collection=https://stac.eurac.edu/collections/SENTINEL2_L2A_SAMPLE \ + --process-levels=L1,L2 +``` + + + + ## Reporting Being a `pytest` based test suite, various plugins can be used to generate reports in different formats as desired. diff --git a/src/openeo_test_suite/tests/workflows/README.md b/src/openeo_test_suite/tests/workflows/README.md index 702de68..75fe713 100644 --- a/src/openeo_test_suite/tests/workflows/README.md +++ b/src/openeo_test_suite/tests/workflows/README.md @@ -1,32 +1,10 @@ # Tests for full openEO workflows (process graphs) -The tests are designed to run using [synchronous calls](https://open-eo.github.io/openeo-python-client/basics.html#download-synchronously) and a Sentinel-2 Collection. -Usage examples: - -```bash -# Compact -pytest --openeo-backend-url=openeo.dataspace.copernicus.eu --s2-collection=SENTINEL2_L2A - -# With full back-end URL, a STAC collection to use instead of a predefined openEO collection -# and a list of process levels to test against -pytest --openeo-backend-url=https://openeo.dataspace.copernicus.eu/openeo/1.2 \ - --s2-collection=https://stac.eurac.edu/collections/SENTINEL2_L2A_SAMPLE \ - --process-levels=L1,L2,L3 -``` - -Parameters: - -- `--s2-collection`: the data collection to test against. It can be either a Sentinel-2 STAC Collection or the name of an openEO Sentinel-2 Collection provided by the back-end. -- `--process-levels`: all process profiles to test against, separated by comma. You need to list all levels explicitly, e.g., L2 does not include L1 automatically. Example: L1,L2,L3. By default tests against all processes. - - -## Notes - -- Only the workflows containing processes that are exposed under the `/processes` endpoint are tested. If `load_stac` is not available, the `--s2-collection` parameter must be set to an existing openEO S2 collection. - -- Tests are divided by process levels, defined [here](https://openeo.org/documentation/1.0/developers/profiles/processes.html). +## Development/status notes +The following notes give a snapshot of the status of the workflow tests +around the development time of this test suite module (Dec 2023). ### VITO @@ -104,10 +82,3 @@ FAILED test_load_save.py::test_load_save_geotiff - openeo.rest.OpenEoApiError: [ ``` ========= 10 failed, 1 error in 13.50s ========== ``` - - -### Additional Info - -The reason behind the creation of two test collections is due to the fact of non-optimal implementation of the `load_stac` process in the various back-ends. -1. CDSE and VITO load the data assigning to the temporal dimension the `t` label and to the bands dimension the `bands` label. To test this back-end it is necessary to use the https://stac.eurac.edu/collections/SENTINEL2_L2A_SAMPLE collection. -2. EURAC and EODC load the data assigning to the temporal dimension the `time` label and to the bands dimension the `band` label. To test this back-end it is necessary to use the https://stac.eurac.edu/collections/SENTINEL2_L2A_SAMPLE_2 collection.