diff --git a/.vuepress/config.js b/.vuepress/config.js index 15355c9b6..b5f634753 100644 --- a/.vuepress/config.js +++ b/.vuepress/config.js @@ -16,12 +16,13 @@ const versions = [ userNav: [ {text: 'Introduction', link: 'index.html'}, {text: 'Glossary', link: 'glossary.html'}, + {text: 'Datacubes', link: 'datacubes.html'}, {text: 'Getting Started', items: [ - {text: 'Datacubes', link: 'datacubes.html'}, {text: 'JavaScript', link: 'javascript/index.html'}, {text: 'Python', link: 'python/index.html'}, {text: 'QGIS', link: 'qgis/index.html'}, - {text: 'R', link: 'r/index.html'} + {text: 'R', link: 'r/index.html'}, + {text: 'Client-Side Processing (Python)', link: 'python/client-side-processing.html'}, ]}, {text: 'Processes', link: 'processes.html'}, {text: 'Cookbook', link: 'cookbook/index.html'}, diff --git a/documentation/1.0/python/client-side-processing.md b/documentation/1.0/python/client-side-processing.md new file mode 100644 index 000000000..c0cf7f3c4 --- /dev/null +++ b/documentation/1.0/python/client-side-processing.md @@ -0,0 +1,191 @@ +# Get started with the openEO Python Client Client Side Processing + +## Background + +The client-side processing functionality allows to test and use openEO with its processes locally, i.e. without any connection to an openEO back-end. +It relies on the projects [openeo-pg-parser-networkx](https://github.com/Open-EO/openeo-pg-parser-networkx>), which provides an openEO process graph parsing tool, and [openeo-processes-dask](https://github.com/Open-EO/openeo-processes-dask), which provides an Xarray and Dask implementation of most openEO processes. + +## Installation + +::: warning Important +This feature requires ``Python>=3.9``. +::: + +The openEO Python client library can easily be installed with a tool like `pip`, for example: + +```shell script +pip install openeo[localprocessing] +``` + + +## Usage + +Every openEO process graph relies on data which is typically provided by a cloud infrastructure (the openEO back-end). +The client-side processing adds the possibility to read and use local netCDFs, geoTIFFs, ZARR files, and remote STAC Collections or Items for your experiments. + +### STAC Collections and Items + +::: warning Important +The provided examples using STAC rely on third party STAC Catalogs, we can't guarantee that the urls will remain valid. +::: + +With the `load_stac` process it's possible to load and use data provided by remote or local STAC Collections or Items. +The following code snippet loads Sentinel-2 L2A data from a public STAC Catalog, using specific spatial and temporal extent, band name and also properties for cloud coverage. + +```python +from openeo.local import LocalConnection +local_conn = LocalConnection("./") + +url = "https://earth-search.aws.element84.com/v1/collections/sentinel-2-l2a" +spatial_extent = {"west": 11, "east": 12, "south": 46, "north": 47} +temporal_extent = ["2019-01-01", "2019-06-15"] +bands = ["red"] +properties = {"eo:cloud_cover": dict(lt=50)} +s2_cube = local_conn.load_stac(url=url, + spatial_extent=spatial_extent, + temporal_extent=temporal_extent, + bands=bands, + properties=properties, +) +s2_cube.execute() +``` + +When calling the `.execute()` method on a `Datacube` created from a `LocalConnection`, an `xarray.DataArray` object containing dask arrays is returned: + +``` + >>> s2_cube.execute() + + dask.array + Coordinates: (12/53) + * time (time) datetime64[ns] 2019-01-02... + id (time) >> # Check if the data is loaded correctly +>>> s2_datacube.execute() + +dask.array +Coordinates: + * t (t) datetime64[ns] 2022-06-02 2022-06-05 ... 2022-06-27 2022-06-30 + * x (x) float64 6.75e+05 6.75e+05 6.75e+05 ... 6.843e+05 6.843e+05 + * y (y) float64 5.155e+06 5.155e+06 5.155e+06 ... 5.148e+06 5.148e+06 + crs |S1 ... + * bands (bands) object 'B04' 'B03' 'B02' 'B08' 'SCL' +Attributes: + Conventions: CF-1.9 + institution: openEO platform - Geotrellis backend: 0.9.5a1 + description: + title: +``` + +As you can see in the previous example, we are using a call to `.execute()` which will execute locally the generated openEO process graph. +In this case, the process graph consist only in a single `load_collection`, which performs lazy loading of the data. With this first step you can check if the data is being read correctly by openEO. + +Looking at the metadata of this netCDF sample, we can see that it contains the bands B04, B03, B02, B08 and SCL. +Additionally, we also see that it is composed by more than one element in time and that it covers the month of June 2022. + +We can now do a simple processing for demo purposes, let's compute the median NDVI in time and visualize the result: + +```python +b04 = s2_datacube.band("B04") +b08 = s2_datacube.band("B08") +ndvi = (b08 - b04) / (b08 + b04) +ndvi_median = ndvi.reduce_dimension(dimension="t", reducer="median") +result_ndvi = ndvi_median.execute() +result_ndvi.plot.imshow(cmap="Greens") +``` + +We can perform the same example using data provided by STAC Collection: + + +```python +from openeo.local import LocalConnection +local_conn = LocalConnection("./") + +url = "https://earth-search.aws.element84.com/v1/collections/sentinel-2-l2a" +spatial_extent = {"east": 11.40, "north": 46.52, "south": 46.46, "west": 11.25} +temporal_extent = ["2022-06-01", "2022-06-30"] +bands = ["red", "nir"] +properties = {"eo:cloud_cover": dict(lt=80)} +s2_datacube = local_conn.load_stac( + url=url, + spatial_extent=spatial_extent, + temporal_extent=temporal_extent, + bands=bands, + properties=properties, +) + +b04 = s2_datacube.band("red") +b08 = s2_datacube.band("nir") +ndvi = (b08 - b04) / (b08 + b04) +ndvi_median = ndvi.reduce_dimension(dimension="time", reducer="median") +result_ndvi = ndvi_median.execute() +``` + +## Client-Side Processing Example Notebooks + +* [From the openEO Python Client repo](https://github.com/Open-EO/openeo-python-client/tree/master/examples/notebooks/Client_Side_Processing) +* [From the Cubes and Clouds repo](https://github.com/EO-College/cubes-and-clouds/blob/main/lectures/3.1_data_processing/exercises/_alternatives/31_data_processing_stac.ipynb) + +## Additional Information + +Additional information and resources about the openEO Python Client Library: + +* [Official openEO Python Client Library Documentation](https://open-eo.github.io/openeo-python-client/) +* [Example Python scripts](https://github.com/Open-EO/openeo-python-client/tree/master/examples) +* [Example Jupyter Notebooks](https://github.com/Open-EO/openeo-python-client/tree/master/examples/notebooks) +* [Repository on GitHub](https://github.com/Open-EO/openeo-python-client)