Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Structured zarr dtype as bytes #176

Open
wants to merge 46 commits into
base: master
Choose a base branch
from

Conversation

brian-michell
Copy link

Hello Tensorstore community. I have a project that requires the ability to open structured data as void #125. We accidentally submitted our hacky workaround #127 but received good feedback suggesting a pure Zarr driver update rather than editing the kvs_chunked_driver.

I've been working on an implementation to eventually submit back into mainline Tensorstore but I've run into some issues.

  1. Handling the extra "byte" dimension of the store transform has spilled into the main driver tensorstore/driver/driver.cc. I hope this is simply my error in missing where it should be handled in the Zarr driver.
  2. Currently, on opening an existing structured Zarr there is no transform which means it does not use my workaround for the above issue. It manifests in tensorstore_driver_zarr_driver_test.cc:1151.
  3. This behavior change impacts tensorstore_driver_zarr_dtype_test.cc, which I would expect insinuates breaking changes to existing code using the ZarrDType class.

I'm hoping to either get some advice on how to proceed or kick off the discussion for the more proper way to implement the behavior.

I also acknowledge the styling and naming conventions have not yet been followed and that the code is not yet optimal, my intent was to fix that upon a proper pull request.

Copy link

google-cla bot commented Jul 9, 2024

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@brian-michell brian-michell marked this pull request as ready for review July 9, 2024 20:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant