Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ Implement cog3pio xarray BackendEntrypoint #14

Merged
merged 10 commits into from
May 9, 2024
Merged

Conversation

weiji14
Copy link
Owner

@weiji14 weiji14 commented Mar 25, 2024

Allow 'cog3pio' to be used as an engine in xarray.open_dataarray and xarray.open_dataset!

Usage (Python):

import xarray as xr

with xr.open_dataarray(
    filename_or_obj="https://github.com/cogeotiff/rio-tiler/raw/6.4.1/tests/fixtures/cog_nodata_nan.tif",
    engine="cog3pio",
) as da:
    assert da.sizes == {'band': 1, 'y': 549, 'x': 549}
    assert da.dtype == "float32"

TODO:

TODO in the future:

  • Handle NaN values in GeoTIFF properly

References:

Initial implementation of a 'cog3pio' xarray BackendEntrypoint for decoding GeoTIFF files! Following instructions at https://docs.xarray.dev/en/v2024.02.0/internals/how-to-add-new-backend.html on registering a backend. Only a minimal implementation for now to read the pixel array data, with dummy x and y coordinates.
@weiji14 weiji14 added the feature New feature or request label Mar 25, 2024
@weiji14 weiji14 added this to the 0.1.0 milestone Mar 25, 2024
@weiji14 weiji14 self-assigned this Mar 25, 2024
weiji14 added 4 commits March 26, 2024 11:40
N-D labeled arrays and datasets in Python!
Work out the list of x and y coordinates for the raster grid from the Affine transformation matrix. Requires changes being developed at georust/geo#1159.
Getting an error with compiling pandas 2.2.1 on Windows x86 (32-bit), and pandas has moved away from providing 32-bit Windows wheels, see pandas-dev/pandas#54979, so might as well not build cog3pio wheels for 32-bit Windows since we can't test them properly.
Parametrized test to compare loading a GeoTIFF using cog3pio vs rioxarray via their respective xarray BackendEntrypoints. Made a new extras dependency group in pyproject.toml called 'benchmark', and listed both `pytest-codspeed` and `rioxarray` under it.
weiji14 added 5 commits March 26, 2024 17:20
The affine transform from the GeoTIFF represents the top-left corner of the top-left pixel (gridline registration), but we need to convert that to the center of the top-left pixel (pixel registration) which is what xarray typically assumes. Added some more comments on how the xy_coords is calculated, and updated unit tests with new x/y min/max bounds. Commented out the mean value assertion for now as NaN handling is not implemented yet.
Getter methods on AffineTransform implemented in georust/geo#1159 has been merged into the main branch of georust/geo, so no longer need to point to my personal fork. Also fixed a mismatched type when accessing the x_res and y_res attributes.
Show how a GeoTIFF file can be read into an xarray.DataArray object by passing `engine="cog3pio"` to xarray.open_dataarray.
The multiple dtype feature is a little trickier than expected, so will push this out as a medium term goal for Q2 2024. Likely will depend on how the georust/geotiff crate's progress is like.
@weiji14 weiji14 marked this pull request as ready for review May 9, 2024 01:47
@weiji14 weiji14 enabled auto-merge (squash) May 9, 2024 01:47
@weiji14 weiji14 merged commit 1009387 into main May 9, 2024
12 checks passed
@weiji14 weiji14 deleted the xarray_backend branch May 9, 2024 03:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant