Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ndpyramid validation #14

Open
freeman-lab opened this issue Sep 29, 2021 · 1 comment
Open

ndpyramid validation #14

freeman-lab opened this issue Sep 29, 2021 · 1 comment

Comments

@freeman-lab
Copy link
Member

freeman-lab commented Sep 29, 2021

From a discussion with @katamartin , we realized it could be very nice to build a tiny utility (possibly in a standalone package) that validates the structure of a zarr ndpyramid from js.

Key things it would do

  • try to load the group via zarr-js
  • extract pyramid metadata
  • extract coordinates and dimensions
  • validate everything for internal consistency
  • raise any issues

Two key use cases

  • short term, via a CLI or tiny standalone web app, let us inspect all dataset properties before trying to incorporate a dataset into an app (this will be especially useful when it comes to coordinates, and just generally helpful for debugging data formatting issues on the js side)
  • long term, ideally use the same underlying package to do runtime validation and throw much more helpful error messages when datasets are not formatted correctly

cc @jhamman

@freeman-lab freeman-lab changed the title pyramid validation ndpyramid validation Sep 29, 2021
@jhamman
Copy link

jhamman commented Sep 30, 2021

I think this is a great idea. Since we're relying on the array metadata and multiscales spec in consolidated metadata, it may make sense to just write a spec validator for the .zmetadata json. If the spec validates, then a set of functional tests could be run.

FWIW, this would also be a useful test approach in ndpyramid as well. The combination of a validation utility here and a set of unit tests in ndpyramid would be quite a nice combination.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants