Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FEATURE REQUEST: make kerchunk work for everything that can be opened by xr.open_dataset #126

Closed
aaronspring opened this issue Feb 21, 2022 · 4 comments

Comments

@aaronspring
Copy link

aaronspring commented Feb 21, 2022

I really like kerchunk. I started #123 because I wanted to use kerchunk on MPIESM climate model output, which turns out is Netcdf2. kerchunk is built on h5py which I think requires Netcdf4. Are there current efforts to widen the range of input types for kerchunk, i.e. allowing more file types, maybe even these that can be opened by xarray.open_dataset?

Or would this essential mean replicating much of the xarray io again in kerchunk?

@martindurant
Copy link
Member

netCDF4/HDF5 was indeed the first format we have developed (largely thanks fo @ajelenak 's deep knowledge of h5py) and certainly the most fruitful. We fully intend to support more formats, and netCDF-classic (i.e., versions 1-3) is definitely on that list.

( @rsignell-usgs , I don't remember which of your suggested datasets included both netCDF4 and netCDF-classic, please link here)

@martindurant
Copy link
Member

@aaronspring , can you point me to one of these files? Which xarray engine do you normally use to read it, and does it matter?

@aaronspring
Copy link
Author

aaronspring commented Feb 21, 2022

does it matter?

no. I don't specify engine.

Tiny cdf2 File: https://owncloud.gwdg.de/index.php/s/UQpymejGK2Uo15g Or do you need a realistic size file?

That files doesnt work with h5py. But it works after convert to nc4: cdo -f nc4 -copy ifile.nc ifile.nc

@martindurant
Copy link
Member

the netCDF3 module seems to handle the netCDF2 case also. Closing this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants