Skip to content

Commit

Permalink
Move WP6 (workflow) specifics to root readme
Browse files Browse the repository at this point in the history
  • Loading branch information
soxofaan committed Feb 5, 2024
1 parent 86b0c79 commit 2d35b0c
Show file tree
Hide file tree
Showing 2 changed files with 51 additions and 32 deletions.
48 changes: 48 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -270,6 +270,54 @@ which is driven by the `OPENEO_AUTH_METHOD` environment variable:
for instructions on how to complete the authentication flow.
## Test module specifics
Some test modules have specific considerations and options to be aware of.
### WP6 Full process graph execution and validation <a name="WP6-specifics"></a>
These tests are designed to run using synchronous openEO process calls
and a Sentinel-2(-like) collection.
The S2 collection to use must be specified through the `--s2-collection` option,
which supports two forms:
- a normal openEO collection name (e.g. `--s2-collection=SENTINEL2_L2A`)
which will be loaded through the standard openEO `load_collection` process.
- a STAC URL (typically starting with `https://`),
which will be loaded with `load_stac`.
The following two example STAC collections are provided in the context of this test suite project.
Each of these exists to cater to subtle differences between some back-end implementations
regarding temporal and band dimension naming and how that is handled in `load_stac`.
- https://stac.eurac.edu/collections/SENTINEL2_L2A_SAMPLE:
defines `"t"` as temporal dimension name, and `"bands"` as bands dimension name.
Recommended to be used with VITO/CDSE openEO backends.
- https://stac.eurac.edu/collections/SENTINEL2_L2A_SAMPLE_2:
uses `"time"` as temporal dimension name, and `"band"` as bands dimension name.
Recommended to be used with EURAC and EODC openEO backends.
If the back-end does not support `load_stac`, a collection name must be used.
#### Usage examples:
```bash
# Compact
pytest src/openeo_test_suite/tests/workflows \
-U openeo.dataspace.copernicus.eu \
--s2-collection=SENTINEL2_L2A
# With full back-end URL, a STAC collection to use instead of a predefined openEO collection
# and a list of process levels to test against
pytest src/openeo_test_suite/tests/workflows \
--openeo-backend-url=https://openeo.dataspace.copernicus.eu/openeo/1.2 \
--s2-collection=https://stac.eurac.edu/collections/SENTINEL2_L2A_SAMPLE \
--process-levels=L1,L2
```
## Reporting
Being a `pytest` based test suite, various plugins can be used to generate reports in different formats as desired.
Expand Down
35 changes: 3 additions & 32 deletions src/openeo_test_suite/tests/workflows/README.md
Original file line number Diff line number Diff line change
@@ -1,32 +1,10 @@
# Tests for full openEO workflows (process graphs)

The tests are designed to run using [synchronous calls](https://open-eo.github.io/openeo-python-client/basics.html#download-synchronously) and a Sentinel-2 Collection.

Usage examples:

```bash
# Compact
pytest --openeo-backend-url=openeo.dataspace.copernicus.eu --s2-collection=SENTINEL2_L2A

# With full back-end URL, a STAC collection to use instead of a predefined openEO collection
# and a list of process levels to test against
pytest --openeo-backend-url=https://openeo.dataspace.copernicus.eu/openeo/1.2 \
--s2-collection=https://stac.eurac.edu/collections/SENTINEL2_L2A_SAMPLE \
--process-levels=L1,L2,L3
```

Parameters:

- `--s2-collection`: the data collection to test against. It can be either a Sentinel-2 STAC Collection or the name of an openEO Sentinel-2 Collection provided by the back-end.
- `--process-levels`: all process profiles to test against, separated by comma. You need to list all levels explicitly, e.g., L2 does not include L1 automatically. Example: L1,L2,L3. By default tests against all processes.


## Notes

- Only the workflows containing processes that are exposed under the `/processes` endpoint are tested. If `load_stac` is not available, the `--s2-collection` parameter must be set to an existing openEO S2 collection.

- Tests are divided by process levels, defined [here](https://openeo.org/documentation/1.0/developers/profiles/processes.html).
## Development/status notes

The following notes give a snapshot of the status of the workflow tests
around the development time of this test suite module (Dec 2023).

### VITO

Expand Down Expand Up @@ -104,10 +82,3 @@ FAILED test_load_save.py::test_load_save_geotiff - openeo.rest.OpenEoApiError: [
```
========= 10 failed, 1 error in 13.50s ==========
```


### Additional Info

The reason behind the creation of two test collections is due to the fact of non-optimal implementation of the `load_stac` process in the various back-ends.
1. CDSE and VITO load the data assigning to the temporal dimension the `t` label and to the bands dimension the `bands` label. To test this back-end it is necessary to use the https://stac.eurac.edu/collections/SENTINEL2_L2A_SAMPLE collection.
2. EURAC and EODC load the data assigning to the temporal dimension the `time` label and to the bands dimension the `band` label. To test this back-end it is necessary to use the https://stac.eurac.edu/collections/SENTINEL2_L2A_SAMPLE_2 collection.

0 comments on commit 2d35b0c

Please sign in to comment.