Issues with aggregate_spatial

[jonathom]

I wanted to use aggregate_spatial in a research project but am running into some problems. The use case is to extract raster values at point locations and return as a vector cube (I understand from processes#37 that this is still pretty much up to aggregate_spatial).

1. It doesn't seem to be able to handle geometries of type linestring or points - only polygons are possible. Fails with "org.locationtech.jts.geom.Point cannot be cast to org.locationtech.jts.geom.Polygon". A workaround with buffering into polygons would of course be possible, but less exact. Example Graph:

{
  "process_graph": {
    "load_collection_HDCDX6606Y": {
      "arguments": {
        "bands": [
          "B04",
          "B03",
          "B02"
        ],
        "id": "SENTINEL2_L2A_SENTINELHUB",
        "spatial_extent": {
          "east": 10.5152,
          "north": 51.3856,
          "south": 51.3371,
          "west": 10.4005
        },
        "temporal_extent": [
          "2018-07-01",
          "2018-10-01"
        ]
      },
      "process_id": "load_collection"
    },
    "aggregate_spatial_VGHHP6336H": {
      "arguments": {
        "context": null,
        "data": {
          "from_node": "load_collection_HDCDX6606Y"
        },
        "geometries": {
          "features": [
            {
              "geometry": {
                "coordinates": [
                  10.4164,
                  51.3701
                ],
                "type": "Point"
              },
              "properties": {
                "POINT_ID": 43503140
              },
              "type": "Feature"
            },
            {
              "geometry": {
                "coordinates": [
                  10.5024,
                  51.3518
                ],
                "type": "Point"
              },
              "properties": {
                "POINT_ID": 43563138
              },
              "type": "Feature"
            },
            {
              "geometry": {
                "coordinates": [
                  10.4737,
                  51.3519
                ],
                "type": "Point"
              },
              "properties": {
                "POINT_ID": 43543138
              },
              "type": "Feature"
            },
            {
              "geometry": {
                "coordinates": [
                  10.4739,
                  51.3699
                ],
                "type": "Point"
              },
              "properties": {
                "POINT_ID": 43543140
              },
              "type": "Feature"
            },
            {
              "geometry": {
                "coordinates": [
                  10.5026,
                  51.3698
                ],
                "type": "Point"
              },
              "properties": {
                "POINT_ID": 43563140
              },
              "type": "Feature"
            },
            {
              "geometry": {
                "coordinates": [
                  10.4163,
                  51.3521
                ],
                "type": "Point"
              },
              "properties": {
                "POINT_ID": 43503138
              },
              "type": "Feature"
            },
            {
              "geometry": {
                "coordinates": [
                  10.445,
                  51.352
                ],
                "type": "Point"
              },
              "properties": {
                "POINT_ID": 43523138
              },
              "type": "Feature"
            },
            {
              "geometry": {
                "coordinates": [
                  10.4452,
                  51.37
                ],
                "type": "Point"
              },
              "properties": {
                "POINT_ID": 43523140
              },
              "type": "Feature"
            }
          ],
          "type": "FeatureCollection"
        },
        "reducer": {
          "process_graph": {
            "mean_WOKMX8709B": {
              "arguments": {
                "data": {
                  "from_parameter": "data"
                },
                "ignore_nodata": true
              },
              "process_id": "mean",
              "result": true
            }
          }
        },
        "target_dimension": "result"
      },
      "process_id": "aggregate_spatial"
    },
    "save_result_BBVGD1016G": {
      "arguments": {
        "data": {
          "from_node": "aggregate_spatial_VGHHP6336H"
        },
        "format": "CSV",
        "options": {}
      },
      "process_id": "save_result",
      "result": true
    }
  }
}

2. aggregate_spatial doesn't seem to work when no t dimension is present. I am not sure whether this is definition compliant or not. Fails with error "'SpatialKey' object has no attribute 'instant'". I tried to simply add back a temporal dimension after reducing it, via add_dimension, but that didn't work either. Example Graph:

{
  "process_graph": {
    "1": {
      "process_id": "reduce_dimension",
      "arguments": {
        "data": {
          "from_node": "load_collection_HDCDX6606Y"
        },
        "reducer": {
          "process_graph": {
            "1": {
              "process_id": "mean",
              "arguments": {
                "data": {
                  "from_parameter": "data"
                }
              },
              "result": true
            }
          }
        },
        "dimension": "t"
      }
    },
    "load_collection_HDCDX6606Y": {
      "arguments": {
        "bands": [
          "B04",
          "B03",
          "B02"
        ],
        "id": "SENTINEL2_L2A_SENTINELHUB",
        "spatial_extent": {
          "east": 10.5152,
          "north": 51.3856,
          "south": 51.3371,
          "west": 10.4005
        },
        "temporal_extent": [
          "2021-07-01T00:00:00Z",
          "2021-07-08T00:00:00Z"
        ]
      },
      "process_id": "load_collection"
    },
    "aggregate_spatial_DHEEQ0430X": {
      "arguments": {
        "context": null,
        "data": {
          "from_node": "1"
        },
        "geometries": {
          "features": [
            {
              "geometry": {
                "coordinates": [
                  [
                    [
                      10.4566,
                      51.3747
                    ],
                    [
                      10.4335,
                      51.3732
                    ],
                    [
                      10.4527,
                      51.3615
                    ],
                    [
                      10.4566,
                      51.3747
                    ]
                  ]
                ],
                "type": "Polygon"
              },
              "properties": {
                "ID": 43562
              },
              "type": "Feature"
            },
            {
              "geometry": {
                "coordinates": [
                  [
                    [
                      10.4566,
                      51.3747
                    ],
                    [
                      10.4334,
                      51.3732
                    ],
                    [
                      10.4528,
                      51.3614
                    ],
                    [
                      10.4566,
                      51.3747
                    ]
                  ]
                ],
                "type": "Polygon"
              },
              "properties": {
                "ID": 745
              },
              "type": "Feature"
            }
          ],
          "type": "FeatureCollection"
        },
        "reducer": {
          "process_graph": {
            "mean_DKRIP6875M": {
              "arguments": {
                "data": {
                  "from_parameter": "data"
                },
                "ignore_nodata": true
              },
              "process_id": "mean",
              "result": true
            }
          }
        },
        "target_dimension": "result"
      },
      "process_id": "aggregate_spatial"
    },
    "save_result_ERWXB2557M": {
      "arguments": {
        "data": {
          "from_node": "aggregate_spatial_DHEEQ0430X"
        },
        "format": "CSV",
        "options": {}
      },
      "process_id": "save_result",
      "result": true
    }
  }
}

3. Attributes of the passed geometries are ignored and not passed along (see graph above). I was hoping to use ID's to merge the extracted pixel values with other data later on in the workflow. Is there any (other) way to accomplish this? I think the definition doesn't mention it at all so I am not sure whether this is also an issue for openeo-processes, (also @m-mohr)?

[m-mohr]

Re 3: In general, it would be very useful to keep the GeoJSON properties so that people can actually identify the Features they've provided in the results. I'm not sure if the processes describe this well enough, but we can probably clarify this in general for vector file formats that handle properties.

[soxofaan]

I think it's best to submit these independent issues separately, otherwise this thread will become unmanageable .

Quick feedback though:

1. known issue: geopyspark driver only support aggregate_spatial with polygons, not points or lines. The current recommended workaround is indeed to apply buffering to points and lines. FYI: we recently added an experimental process vector_buffer in EP-3884 vector buffer openeo-python-driver#76 for doing the buffering backend-side.
2. this looks like a bug
3. known issue: in python driver we eliminate feature properties quite early (see Deprecate GeometryCollection in favor of FeatureCollection (e.g. DelayedVector) openeo-python-driver#71). It's good idea to have some way to preserve original properties in feature collections. However, as @m-mohr noted this behavior has to be spec'ed out a bit more.

[jonathom]

Alright, thanks for the feedback! Opened the above issue for the bug.

[soxofaan]

ok thanks, it's best to close this one then.