Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

VectorCube support #114

Open
soxofaan opened this issue Apr 20, 2022 · 3 comments
Open

VectorCube support #114

soxofaan opened this issue Apr 20, 2022 · 3 comments
Labels
vector cubes

Comments

@soxofaan
Copy link
Member

VectorCubes are one of the important new concepts at the moment.
Adding full support is probably going to take a couple of iterations, so this is probably going to be an epic/grouping ticket.

related/subtasks:
soxofaan added a commit that referenced this issue Apr 20, 2022
@soxofaan
EP-3981 Add #71 #114 references to vector cube todo's
3520676
soxofaan added a commit that referenced this issue Apr 20, 2022
@soxofaan
EP-3981 #114 add tests for aggregate_spatial with vector cubes
1fba8f8
@soxofaan soxofaan mentioned this issue Apr 20, 2022
Merged
soxofaan added a commit that referenced this issue Apr 20, 2022
@soxofaan
EP-3981 Add #71 #114 references to vector cube todo's
7d540bb
soxofaan added a commit that referenced this issue Apr 20, 2022
@soxofaan
EP-3981 #114 add tests for aggregate_spatial with vector cubes
1b0902e
soxofaan added a commit that referenced this issue Apr 20, 2022
@soxofaan
EP-3981 #114 add more DriverVectorCube tests
9d17693
@soxofaan
Copy link
Member Author

  • implemented vector cube based aggregate_spatial (in python driver's DummyDataCube)

soxofaan added a commit that referenced this issue Apr 20, 2022
@soxofaan
Removed old, unused zonal_statistics stubs from openeo-python-driver
f925ae5 
related to #114
soxofaan added a commit that referenced this issue Apr 22, 2022
@soxofaan
Bump version to 0.18.1a1
4a4bb5b 
related to EP-3981 #114
soxofaan added a commit that referenced this issue Apr 22, 2022
@soxofaan
EP-3981 #114 DriverVectorCube: add from_geojson and to_wkt
ec74ca3
soxofaan added a commit that referenced this issue Apr 22, 2022
@soxofaan
EP-3981 #114 add to_vector_cube process (custom) for converting Geo…
927a115 
…Json to VectorCube
soxofaan added a commit that referenced this issue Apr 22, 2022
@soxofaan
EP-3981 #114 Bump version to 0.18.2a1
200a3df
@soxofaan soxofaan mentioned this issue Jun 1, 2022
Open
@soxofaan soxofaan mentioned this issue Sep 13, 2022
Closed
soxofaan added a commit that referenced this issue Oct 5, 2022
@soxofaan
Issue #114/#141 convert inline GeoJSON in aggregate_spatial to Vector…
0d68104 
…Cube
soxofaan added a commit that referenced this issue Oct 5, 2022
@soxofaan
fixup! Issue #114/#141 convert inline GeoJSON in aggregate_spatial to…
886b663 
… VectorCube
soxofaan added a commit that referenced this issue Oct 6, 2022
@soxofaan
fixup! Issue #114/#141 convert inline GeoJSON in aggregate_spatial to…
4a6d5ea 
… VectorCube
soxofaan added a commit that referenced this issue Oct 6, 2022
@soxofaan
fixup! fixup! Issue #114/#141 convert inline GeoJSON in aggregate_spa…
2864f25 
…tial to VectorCube
soxofaan added a commit that referenced this issue Oct 6, 2022
@soxofaan
fixup! fixup! fixup! Issue #114/#141 convert inline GeoJSON in aggreg…
f85e929 
…ate_spatial to VectorCube
soxofaan added a commit that referenced this issue Oct 6, 2022
@soxofaan
fixup! fixup! fixup! fixup! Issue #114/#141 convert inline GeoJSON in…
bce577b 
… aggregate_spatial to VectorCube
soxofaan added a commit that referenced this issue Oct 7, 2022
@soxofaan
fixup! fixup! fixup! fixup! fixup! Issue #114/#141 convert inline Geo…
73ae035 
…JSON in aggregate_spatial to VectorCube
soxofaan added a commit that referenced this issue Oct 7, 2022
@soxofaan
Issue #114/#141 convert inline GeoJSON in aggregate_spatial to Vector…
c8fcd91 
…Cube
soxofaan added a commit that referenced this issue Oct 7, 2022
@soxofaan
fixup! Issue #114/#141 convert inline GeoJSON in aggregate_spatial to…
a27cc9b 
… VectorCube
soxofaan added a commit that referenced this issue Oct 7, 2022
@soxofaan
fixup! Issue #114/#141 convert inline GeoJSON in aggregate_spatial to…
4e02771 
… VectorCube
soxofaan added a commit that referenced this issue Oct 7, 2022
@soxofaan
fixup! fixup! Issue #114/#141 convert inline GeoJSON in aggregate_spa…
34cad31 
…tial to VectorCube
soxofaan added a commit that referenced this issue Oct 7, 2022
@soxofaan
fixup! fixup! fixup! Issue #114/#141 convert inline GeoJSON in aggreg…
a77d50a 
…ate_spatial to VectorCube
soxofaan added a commit that referenced this issue Oct 7, 2022
@soxofaan
fixup! fixup! fixup! fixup! Issue #114/#141 convert inline GeoJSON in…
f43b98b 
… aggregate_spatial to VectorCube
soxofaan added a commit that referenced this issue Oct 7, 2022
@soxofaan
fixup! fixup! fixup! fixup! fixup! Issue #114/#141 convert inline Geo…
842b3d7 
…JSON in aggregate_spatial to VectorCube
soxofaan added a commit to Open-EO/openeo-geopyspark-driver that referenced this issue Oct 12, 2022
@soxofaan
Add DriverVectorCube support to batch_job.py: extract_result_metadata
8a6b3a4 
ref: Open-EO/openeo-python-driver#114, Open-EO/openeo-python-driver#141
soxofaan added a commit to Open-EO/openeo-geopyspark-integrationtests that referenced this issue Oct 12, 2022
@soxofaan
Update test_random_forest_train_and_load_from_jobid
9a5bbac 
ref: Open-EO/openeo-python-driver#114, Open-EO/openeo-python-driver#141
soxofaan added a commit to Open-EO/openeo-geopyspark-driver that referenced this issue Oct 12, 2022
@soxofaan
Improve DriverVectorCube support in batch_job.py: extract_result_meta…
aac7d4b 
…data

ref: Open-EO/openeo-python-driver#114, Open-EO/openeo-python-driver#141
soxofaan added a commit that referenced this issue Oct 12, 2022
@soxofaan
fixup! Issue #114/#141 temporarily roll back automatic conversion of …
4927eee 
…inline Point GeoJSON to VectorCube in aggregate_spatial
soxofaan added a commit that referenced this issue Oct 18, 2022
@soxofaan
Issue #114/#141/#239 re-enable automatic vector cube conversion for P…
4400565 
…oint geometries in aggregate_spatial

reverts 2ec255e and 4927eee
openeo-geopyspark-driver now properly supports point geometries
@soxofaan soxofaan mentioned this issue Oct 26, 2022
Open
jdries added a commit that referenced this issue Mar 29, 2023
@jdries
support vector cube in buffer_vector
1d98c88 
#114
soxofaan added a commit that referenced this issue Aug 2, 2023
@soxofaan
Issue #114/#200 support promoting feature properties to cube values
082b812 
- flattening: move options to export phase iso vector cube constructor
- introduce `VectorCube.from_geodataframe` wiith support for promoting selected columns to cube values
- regardless of promotion: all properties are still associated with `VectorCube.geometries` for now (otherwise properties can not be preserved when using `aggregate_spatial`, see Open-EO/openeo-api#504)
- only promote numerical values by default for now
soxofaan added a commit that referenced this issue Aug 3, 2023
@soxofaan
Issue #114/#200 support promoting feature properties to cube values
c2842ed 
- flattening: move options to export phase iso vector cube constructor
- introduce `VectorCube.from_geodataframe` wiith support for promoting selected columns to cube values
- regardless of promotion: all properties are still associated with `VectorCube.geometries` for now (otherwise properties can not be preserved when using `aggregate_spatial`, see Open-EO/openeo-api#504)
- only promote numerical values by default for now
soxofaan added a commit that referenced this issue Aug 3, 2023
@soxofaan
fixup! Issue #114/#200 support promoting feature properties to cube v…
4bdd41f 
…alues
soxofaan added a commit that referenced this issue Aug 3, 2023
@soxofaan
fixup! Issue #114/#200 support promoting feature properties to cube v…
8f90397 
…alues
soxofaan added a commit that referenced this issue Aug 3, 2023
@soxofaan
Issue #114/#200 support promoting feature properties to cube values
090efca 
- flattening: move options to export phase iso vector cube constructor
- introduce `VectorCube.from_geodataframe` wiith support for promoting selected columns to cube values
- regardless of promotion: all properties are still associated with `VectorCube.geometries` for now (otherwise properties can not be preserved when using `aggregate_spatial`, see Open-EO/openeo-api#504)
- only promote numerical values by default for now
soxofaan added a commit that referenced this issue Aug 4, 2023
@soxofaan
Issue #114/#211/#197 tests for load_geojson and its properties arg
c51cea4
soxofaan added a commit that referenced this issue Aug 4, 2023
@soxofaan
fixup! Issue #114/#211/#197 tests for load_geojson and its `propert…
93aebdd 
…ies` arg
soxofaan added a commit that referenced this issue Aug 4, 2023
@soxofaan
Issue #114/#211/#197 from_geodataframe: add dummy cube when no proper…
9fb130a 
…ties are specified
soxofaan added a commit that referenced this issue Aug 4, 2023
@soxofaan
fixup! Issue #114/#211/#197 from_geodataframe: add dummy cube when no…
2c89fb0 
… properties are specified
soxofaan added a commit that referenced this issue Aug 4, 2023
@soxofaan
fixup! fixup! Issue #114/#211/#197 from_geodataframe: add dummy cube …
477a301 
…when no properties are specified
soxofaan added a commit that referenced this issue Aug 4, 2023
@soxofaan
Issue #114/#211/#197 tests for load_geojson and its properties arg
69cf166
soxofaan added a commit that referenced this issue Aug 4, 2023
@soxofaan
Issue #114/#211/#197 from_geodataframe: add dummy cube when no proper…
1dd06e3 
…ties are specified
soxofaan added a commit that referenced this issue Aug 4, 2023
@soxofaan
Issue #114/#211/#197 bump version to 0.61.0a1
d0b3e8a
soxofaan added a commit that referenced this issue Aug 4, 2023
@soxofaan
Issue #114/#211/#197 bump version to 0.61.0a1
ec8184b
@soxofaan soxofaan mentioned this issue Aug 7, 2023
Merged
soxofaan added a commit that referenced this issue Aug 7, 2023
@soxofaan
Issue #114/#211 initial support for load_url (vector cubes)
ab2ad96
soxofaan added a commit that referenced this issue Aug 8, 2023
@soxofaan
Issue #114/#211/#197 vector cube apply_dimension: put results in cu…
c862e1b 
…be (too)
@soxofaan soxofaan mentioned this issue Dec 15, 2023
Closed
@JeroenVerstraelen JeroenVerstraelen mentioned this issue Jan 31, 2024
Closed
@JeroenVerstraelen
Copy link
Contributor

As discussed, I think we would benefit in terms of future complexity if DriverVectorCube._geometries can be reduced to a gpd.GeoSeries instead of a gpd.GeoDataFrame. This means that DriverVectorCube._cube would have dimensions [geometry, time (optional), bands (optional)] and all property data is loaded in as separate bands.

I would also argue to not allow a "properties" dimension in DriverVectorCube._cube, but to always refer to it as "bands".

@soxofaan
Copy link
Member Author

soxofaan commented Feb 2, 2024

DriverVectorCube._geometries can be reduced to a gpd.GeoSeries instead of a gpd.GeoDataFrame

FYI: using a GeoDataFrame in DriverVectorCube in addition to a DataArray for the "cube data" was/is intentional to separate geometry/feature properties from the cube data, based on initial brainstorming on vector cubes. Now there is some consensus to cast properties to the cube data to simplify things, but that might complicate other things as well: e.g. see

This means that DriverVectorCube._cube would have dimensions [geometry, time (optional), bands (optional)] and all property data is loaded in as separate bands.

note that DriverVectorCube._cube probably also still has to be nullable (e.g. value None) when there is no cube data. For example when loading a single GeoJSON Polygon with load_geojson. Related:

I would also argue to not allow a "properties" dimension in DriverVectorCube._cube, but to always refer to it as "bands".

Note that load_geojson at the moment explicitly specifies usage of the "properties" dimension of type "other":
https://github.com/Open-EO/openeo-processes/blob/965bbaebd4d5984203a0437076c85a66a72a23e0/proposals/load_geojson.json#L21
So we have to be careful when deciding to deviate from that current spec (e.g. first try to push for a different convention in the spec). Related:

soxofaan added a commit that referenced this issue Mar 14, 2024
@soxofaan
#71 #114 #268 added TODO notes on deprecation of GeometryCollection
af58ed2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
vector cubes
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@soxofaan @JeroenVerstraelen