Add client side processing

.vuepress/config.js

@@ -16,12 +16,13 @@ const versions = [
     userNav: [
       {text: 'Introduction', link: 'index.html'},
       {text: 'Glossary', link: 'glossary.html'},
+      {text: 'Datacubes', link: 'datacubes.html'},
       {text: 'Getting Started', items: [
-        {text: 'Datacubes', link: 'datacubes.html'},
         {text: 'JavaScript', link: 'javascript/index.html'},
         {text: 'Python', link: 'python/index.html'},
         {text: 'QGIS', link: 'qgis/index.html'},
-        {text: 'R', link: 'r/index.html'}
+        {text: 'R', link: 'r/index.html'},
+        {text: 'Client-Side Processing (Python)', link: 'python/client-side-processing.html'},
       ]},
       {text: 'Processes', link: 'processes.html'},
       {text: 'Cookbook', link: 'cookbook/index.html'},

documentation/1.0/python/client-side-processing.md

@@ -0,0 +1,191 @@
+# Get started with the openEO Python Client Client Side Processing
+
+## Background
+
+The client-side processing functionality allows to test and use openEO with its processes locally, i.e. without any connection to an openEO back-end.
+It relies on the projects [openeo-pg-parser-networkx](https://github.com/Open-EO/openeo-pg-parser-networkx>), which provides an openEO process graph parsing tool, and [openeo-processes-dask](https://github.com/Open-EO/openeo-processes-dask), which provides an Xarray and Dask implementation of most openEO processes.
+
+## Installation
+
+::: warning Important
+This feature requires ``Python>=3.9``.
+:::
+
+The openEO Python client library can easily be installed with a tool like `pip`, for example:
+
+```shell script
+pip install openeo[localprocessing]
+```
+
+
+## Usage
+
+Every openEO process graph relies on data which is typically provided by a cloud infrastructure (the openEO back-end).
+The client-side processing adds the possibility to read and use local netCDFs, geoTIFFs, ZARR files, and remote STAC Collections or Items for your experiments.
+
+### STAC Collections and Items
+
+::: warning Important
+The provided examples using STAC rely on third party STAC Catalogs, we can't guarantee that the urls will remain valid.
+:::
+
+With the `load_stac` process it's possible to load and use data provided by remote or local STAC Collections or Items.
+The following code snippet loads Sentinel-2 L2A data from a public STAC Catalog, using specific spatial and temporal extent, band name and also properties for cloud coverage.
+
+```python
+from openeo.local import LocalConnection
+local_conn = LocalConnection("./")
+
+url = "https://earth-search.aws.element84.com/v1/collections/sentinel-2-l2a"
+spatial_extent = {"west": 11, "east": 12, "south": 46, "north": 47}
+temporal_extent = ["2019-01-01", "2019-06-15"]
+bands = ["red"]
+properties = {"eo:cloud_cover": dict(lt=50)}
+s2_cube = local_conn.load_stac(url=url,
+    spatial_extent=spatial_extent,
+    temporal_extent=temporal_extent,
+    bands=bands,
+    properties=properties,
+)
+s2_cube.execute()
+```
+
+When calling the `.execute()` method on a `Datacube` created from a `LocalConnection`, an `xarray.DataArray` object containing dask arrays is returned:
+
+```
+    >>> s2_cube.execute()
+    <xarray.DataArray 'stackstac-08730b1b5458a4ed34edeee60ac79254' (time: 177,
+                                                                    band: 1,
+                                                                    y: 11354,
+                                                                    x: 8025)>
+    dask.array<getitem, shape=(177, 1, 11354, 8025), dtype=float64, chunksize=(1, 1, 1024, 1024), chunktype=numpy.ndarray>
+    Coordinates: (12/53)
+      * time                                     (time) datetime64[ns] 2019-01-02...
+        id                                       (time) <U24 'S2B_32TPR_20190102_...
+      * band                                     (band) <U3 'red'
+      * x                                        (x) float64 6.52e+05 ... 7.323e+05
+      * y                                        (y) float64 5.21e+06 ... 5.096e+06
+        s2:product_uri                           (time) <U65 'S2B_MSIL2A_20190102...
+        ...                                       ...
+        raster:bands                             object {'nodata': 0, 'data_type'...
+        gsd                                      int32 10
+        common_name                              <U3 'red'
+        center_wavelength                        float64 0.665
+        full_width_half_max                      float64 0.038
+        epsg                                     int32 32632
+    Attributes:
+        spec:        RasterSpec(epsg=32632, bounds=(600000.0, 4990200.0, 809760.0...
+        crs:         epsg:32632
+        transform:   | 10.00, 0.00, 600000.00|\n| 0.00,-10.00, 5300040.00|\n| 0.0...
+        resolution:  10.0
+```
+
+### Local Collections
+
+If you want to use our sample data, please clone this repository:
+
+```bash
+git clone https://github.com/Open-EO/openeo-localprocessing-data.git
+```
+
+With some sample data we can now check the STAC metadata for the local files by doing:
+
+```python
+from openeo.local import LocalConnection
+local_data_folders = [
+   "./openeo-localprocessing-data/sample_netcdf",
+   "./openeo-localprocessing-data/sample_geotiff",
+]
+local_conn = LocalConnection(local_data_folders)
+local_conn.list_collections()
+```
+This code will parse the metadata content of each netCDF, geoTIFF or ZARR file in the provided folders and return a JSON object containing the STAC representation of the metadata.
+If this code is run in a Jupyter Notebook, the metadata will be rendered nicely.
+
+::: tip
+The code expects local files to have a similar structure to the sample files provided [here](https://github.com/Open-EO/openeo-localprocessing-data.git).
+If the code can not handle you special netCDF, you can still modify the function that reads the metadata from it [here](https://github.com/Open-EO/openeo-python-client/blob/master/openeo/local/collections.py) and the function that reads the data [here](https://github.com/Open-EO/openeo-python-client/blob/master/openeo/local/processing.py).
+:::
+
+### Local Processing
+
+Let's start with the provided sample netCDF of Sentinel-2 data:
+```python
+local_collection = "openeo-localprocessing-data/sample_netcdf/S2_L2A_sample.nc"
+s2_datacube = local_conn.load_collection(local_collection)
+```
+```
+>>> # Check if the data is loaded correctly
+>>> s2_datacube.execute()
+<xarray.DataArray (bands: 5, t: 12, y: 705, x: 935)>
+dask.array<stack, shape=(5, 12, 705, 935), dtype=float32, chunksize=(1, 12, 705, 935), chunktype=numpy.ndarray>
+Coordinates:
+  * t        (t) datetime64[ns] 2022-06-02 2022-06-05 ... 2022-06-27 2022-06-30
+  * x        (x) float64 6.75e+05 6.75e+05 6.75e+05 ... 6.843e+05 6.843e+05
+  * y        (y) float64 5.155e+06 5.155e+06 5.155e+06 ... 5.148e+06 5.148e+06
+    crs      |S1 ...
+  * bands    (bands) object 'B04' 'B03' 'B02' 'B08' 'SCL'
+Attributes:
+    Conventions:  CF-1.9
+    institution:  openEO platform - Geotrellis backend: 0.9.5a1
+    description:
+    title:
+```
+
+As you can see in the previous example, we are using a call to `.execute()` which will execute locally the generated openEO process graph.
+In this case, the process graph consist only in a single `load_collection`, which performs lazy loading of the data. With this first step you can check if the data is being read correctly by openEO.
+
+Looking at the metadata of this netCDF sample, we can see that it contains the bands B04, B03, B02, B08 and SCL.
+Additionally, we also see that it is composed by more than one element in time and that it covers the month of June 2022.
+
+We can now do a simple processing for demo purposes, let's compute the median NDVI in time and visualize the result:
+
+```python
+b04 = s2_datacube.band("B04")
+b08 = s2_datacube.band("B08")
+ndvi = (b08 - b04) / (b08 + b04)
+ndvi_median = ndvi.reduce_dimension(dimension="t", reducer="median")
+result_ndvi = ndvi_median.execute()
+result_ndvi.plot.imshow(cmap="Greens")
+```
+
+We can perform the same example using data provided by STAC Collection:
+
+
+```python
+from openeo.local import LocalConnection
+local_conn = LocalConnection("./")
+
+url = "https://earth-search.aws.element84.com/v1/collections/sentinel-2-l2a"
+spatial_extent =  {"east": 11.40, "north": 46.52, "south": 46.46, "west": 11.25}
+temporal_extent = ["2022-06-01", "2022-06-30"]
+bands = ["red", "nir"]
+properties = {"eo:cloud_cover": dict(lt=80)}
+s2_datacube = local_conn.load_stac(
+    url=url,
+    spatial_extent=spatial_extent,
+    temporal_extent=temporal_extent,
+    bands=bands,
+    properties=properties,
+)
+
+b04 = s2_datacube.band("red")
+b08 = s2_datacube.band("nir")
+ndvi = (b08 - b04) / (b08 + b04)
+ndvi_median = ndvi.reduce_dimension(dimension="time", reducer="median")
+result_ndvi = ndvi_median.execute()
+```
+
+## Client-Side Processing Example Notebooks
+
+* [From the openEO Python Client repo](https://github.com/Open-EO/openeo-python-client/tree/master/examples/notebooks/Client_Side_Processing)
+* [From the Cubes and Clouds repo](https://github.com/EO-College/cubes-and-clouds/blob/main/lectures/3.1_data_processing/exercises/_alternatives/31_data_processing_stac.ipynb)
+
+## Additional Information
+
+Additional information and resources about the openEO Python Client Library:
+
+* [Official openEO Python Client Library Documentation](https://open-eo.github.io/openeo-python-client/)
+* [Example Python scripts](https://github.com/Open-EO/openeo-python-client/tree/master/examples)
+* [Example Jupyter Notebooks](https://github.com/Open-EO/openeo-python-client/tree/master/examples/notebooks)
+* [Repository on GitHub](https://github.com/Open-EO/openeo-python-client)