-
-
Notifications
You must be signed in to change notification settings - Fork 272
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Prune stores #1791
base: v3
Are you sure you want to change the base?
Prune stores #1791
Conversation
* refactor(v3): Using appropriate types * fix(v3): Typing fixes + minor code fixes * fix(v3): _sync_iter works with coroutines * docs(v3/store/core.py): clearer comment * fix(metadata.py): Use Any outside TYPE_CHECKING for Pydantic * fix(zarr/v3): correct zarr format + remove unused method * fix(v3/store/core.py): Potential suggestion on handling str store_like * refactor(zarr/v3): Add more typing * ci(.pre-commit-config.yaml): zarr v3 mypy checks turned on in pre-commit
…elopers#1728) * Specify v3 hatch envs using GitHub actions matrix * Update .github/workflows/test-v3.yml Co-authored-by: Joe Hamman <[email protected]> * Update .github/workflows/test-v3.yml Co-authored-by: Joe Hamman <[email protected]> * test on 3.12 too * no 3.12 --------- Co-authored-by: Joe Hamman <[email protected]> Co-authored-by: Joe Hamman <[email protected]>
* black -> ruff + cleanup * format * Preserve git blame * pre-commit fix
…ntributing.rst (zarr-developers#1643) Co-authored-by: Joe Hamman <[email protected]>
This PR specifically removes following the following v2 stores, and their v3 counterparts: One question that @jhamman raised was whether we want to remove Once we decide on whether or not we keep |
My feeling is that Zip stores are really an underappreciated feature of Zarr. Whatever we choose in terms of design, we should document Zip stores prominently. I'm curious--are there performance or feature differences between the native Zip store and the fsspec version? |
fsspec's ZipFS ought to be able to do everything, I would have thought, with the additional plus of being able to work with remote zips too (usually "r" and "w" modes only). It has certainly not been tested for speed, convenience being the priority over performance. A more specialised, and therefore simple, implementation can be reasonable if there is someone to maintain it. @mxmlnkn , you might be interested by another way of thinking of compressed bunch-of-files containers discussed here. |
Updates:
I think the |
Removes a lot of stores from the v2 codebase. I basically removed every store that required an extra dependency (other than fsspec), and also removed the N5 stores and the sqlite store. The logic for doing this is to thin down the v2 codebase.
Depends on #1742.
TODO: