-
Notifications
You must be signed in to change notification settings - Fork 18
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use Proxy store for nested chunks #85
Conversation
Code adapted from gzuidhof/zarr.js#88 (comment) No Typescript types yet
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Left a small comment. In general, it should be ok to reuse the original open
function since nested only effects chunk keys and some stores used in vizarr will not be nested.
Instead we should try to feature detect when "nested" is needed and just update the underlying "store" for the zarr.Group
.
src/utils.ts
Outdated
function nested(store) { | ||
const get = (target, key) => { | ||
if (key === 'getItem' || key === 'setItem' || key === 'containsItem') { | ||
return (path, ...args) => { | ||
if (path.endsWith('.zarray') || path.endsWith('.zattrs') || path.endsWith('.zgroup')) { | ||
return target[key](path, ...args); | ||
} | ||
const prefix = path.split('/'); | ||
const chunkKey = prefix.pop(); | ||
const newPath = [...prefix, chunkKey.replaceAll('.', '/')].join('/'); | ||
return target[key](newPath, ...args); | ||
} | ||
} | ||
return Reflect.get(target, key); | ||
}; | ||
return new Proxy(store, { get }); | ||
} | ||
const nested_store = nested(store); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would actually move this to be a utility function (outside of open
), since this would break vizarr for folks who don't have a nested store (e.g. imjoy usage, Jupyter notebooks).
My thought is that in io.ts
, we can check for OME-Zarr version and replace the underlying store with a proxy if needed.
if (node instanceof ZarrGroup) {
const attrs = (await node.attrs.asObject()) as Ome.Attrs;
// not sure where the v0.2 meta data is located, but something similar to this.
if (attrs.version === "0.2") {
node.store = nested(node.store); // replaces underlying store with proxied store.
}
if ('plate' in attrs) {
return loadPlate(config, node, attrs.plate);
}
if ('well' in attrs) {
return loadWell(config, node, attrs.well);
}
/* ... */
BTW, I wrote a small CLI today to launch multiple instances of vizarr in different tabs with various data URLs. I found myself opening the images manually from the OME-Blog to test changes (which could get tedious). It's written in Typescript and made to execute in deno, so the script can be used/installed simply via a gist. Install $ deno install --allow-read --allow-net --allow-run https://gist.githubusercontent.com/manzt/f24b30b4f59c3d419ecf1e37dd35ee86/raw/b6e567b4c32090f286e5af71ec445a280a89d7e8/launch-vizarr.ts List all images in configuration $ launch-vizarr --list --config https://raw.githubusercontent.com/ome/blog/master/_data/zarr_data_2020-12-01.json
# ┌───────┬────────┬────────────┬─────────────────────────┬───────────┬───────┬────────┬──────────────┬─────────────────────┬───────────────┐
# │ (idx) │ id │ image_id │ name │ study │ wells │ fields │ acquisitions │ dimensions │ first_version │
# ├───────┼────────┼────────────┼─────────────────────────┼───────────┼───────┼────────┼──────────────┼─────────────────────┼───────────────┤
# │ 0 │ "2551" │ "1229801" │ "JL_120731_S6A" │ "idr0001" │ "96" │ "6" │ "6" │ "1376x1040x16x2x1" │ "0.1" │
# │ 1 │ "422" │ "179693" │ "plate1_1_013" │ "idr0002" │ "96" │ "1" │ "1" │ "1344x1024x1x2x329" │ "0.1" │
# │ 2 │ "1751" │ "692149" │ "P101" │ "idr0004" │ "69" │ "1" │ "1" │ "672x510x10x2x1" │ "0.1" │
# │ 3 │ "5966" │ "3230447" │ "41744_illum_corrected" │ "idr0033" │ "384" │ "9" │ "1" │ "1080x1080x1x5x1" │ "0.1" │
# │ 4 │ "7825" │ "10567921" │ "ESP0025722" │ "idr0094" │ "96" │ "9" │ "1" │ "1080x1080x1x1x1" │ "0.1" │
# └───────┴────────┴────────────┴─────────────────────────┴───────────┴───────┴────────┴──────────────┴─────────────────────┴───────────────┘ Open 2 browser tabes for plates 0 & 4 in a development version of vizarr $ launch-vizarr --vizarr http://localhost:8080 --idx 0,4 --config https://raw.githubusercontent.com/ome/blog/master/_data/zarr_data_2020-12-01.json Both of these JSON files are recognized from the OME-Blog: https://github.com/ome/blog/tree/master/_data, more in the README: https://gist.github.com/manzt/f24b30b4f59c3d419ecf1e37dd35ee86 |
Thanks for the pointers. Should be looking better now, although there may be nicer ways of pleasing the TypeScript Gods with |
d5a6676
to
c2bbed8
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for working on this, Will. I made a few comments. Slight preference to move check / handling of nesting outside of each load*
function and into one place in io.ts
.
Also noticing that linting/formatting GH Actions only occur on my |
@will-moore this looks like it's almost there (nesting logic duplicated now in |
7545a5c
to
b879cdb
Compare
Sorry, missed that. Fixed now. Thanks. |
I'm exporting a nested plate to https://minio-dev.openmicroscopy.org/idr/idr0001-graml-sysgro/pr59_nested/2551.zarr (currently in-progress)... |
Great, thank you! Seeing zarr-developers/zarr-python#715, I'm thinking we will eventually need to create a more sophisticated check for Essentially, vizarr has to handle two OME-Zarr "0.2" variants which I think could get complicated. cc: @joshmoore |
I assume this could also be an issue in python, where |
I'm going to merge this for now, but I'm curious if any have comments on a plan for how |
👍 ome/ome-zarr-py#75 implements a workaround while we wait on |
Code adapted from gzuidhof/zarr.js#88 (comment)
In progress:
ome.ts
and only for OME-Zarr version 0.2)