Ome zarr v0.3 tweaks #104

will-moore · 2021-07-12T14:29:52Z

Use the axes data for dimension names and don't assume data is 5D.

This PR build is deployed at https://deploy-preview-104--vizarr.netlify.app/
These currently all working except 2D example:

joshmoore · 2021-07-13T08:51:38Z

cc: @constantinpape @tischi

manzt · 2021-07-13T14:35:55Z

@will-moore - merged #105

manzt · 2021-07-16T14:07:28Z

Thanks for this PR! @will-moore - is this ready for review?

will-moore · 2021-07-16T22:51:13Z

Yes, I think it's ready for a look. I was hoping to prepare some v0.3 test data and get it public but I ran out of time and now I'm on holidays for 2-3 weeks, so you might want to wait a bit.

manzt · 2021-07-17T19:40:55Z

Great! Enjoy the well deserved holiday 🙂

manzt

Nice! Just a few comments about organization. In general, I'd prefer that all parsing of axes for OME-data lives in ome.ts. Curious to hear your thoughts.

src/ome.ts

manzt · 2021-07-26T02:31:19Z

src/io.ts

@@ -121,6 +110,9 @@ export async function createSourceData(config: ImageLayerConfig): Promise<Source

  if (node instanceof ZarrGroup) {
    const attrs = (await node.attrs.asObject()) as Ome.Attrs;
+    if ('multiscales' in attrs && attrs.multiscales?.[0]?.axes) {
+      config.axis_labels = attrs.multiscales?.[0]?.axes;


I'm not sure what I think about overriding config.axis_labels here. I think we can just inspect the multiscales metadata in loadOmeroMultiscales and derive the axis labels for well and plate from the representative image with multiscales.

I've moved this to ome.ts in 8ed9c89 but this doesn't handle the case when we have axes data but no omero data (so loadOmeroMultiscales() is not called). E.g. sample data at https://oc.embl.de/index.php/s/KLTaSgX1c4zgnHP
Don't know if data with axes should be handled by ome.ts too?

src/ome.ts

src/utils.ts

manzt · 2021-08-05T14:17:18Z

@will-moore can you rebase on main?

git switch main
git pull # pull down latest changes
git switch ome_zarr_v0.3_tweaks
git rebase main # should just auto rebase
npm install # install latest dependencies
git push --force # need to force push to github

hms-dbmi#100 (comment)

will-moore · 2021-08-06T16:02:42Z

I'm seeing an intermittent failure e.g. https://deploy-preview-104--vizarr.netlify.app/?source=https://minio-dev.openmicroscopy.org/idr/v0.3/idr0062-blin-nuclearsegmentation/6001240.zarr/

Sometimes this works:

But even with a refresh, it can fail:

manzt · 2021-08-06T16:22:00Z

Having a look now. I can't seem to reproduce locally with npm run build && npm run preview... I added sourcemaps and unminified the build to hopefully debug better. Curious if its something with netlify preview.

will-moore · 2021-08-09T11:55:30Z

Thanks @manzt. I'm only seeing that error with the netlify preview, so maybe we can ignore or workaround for now. Just didn't want all the broken links above to be a blocker.

The main question that I have now is where to handle the axes info for OME-NGFF that don't have omero data (so aren't handled by loadOmeroMultiscales(). Maybe we do loadOmeroMultiscales() if we have omero OR axes data (but neither are essential)? And this could be renamed to loadOmeMultiscales().
This would mean that utils.loadMultiscales() is never used except for v0.1 and v0.2 data that are also missing omero. And loadOmeMultiscales() could also handle this if it can handle the missing omero and axes data.

I guess I'm saying that we don't need utils.loadMultiscales(), so if that seems OK I'll go ahead and see what that looks like...?

manzt · 2021-08-10T22:33:55Z

Ah, sorry @will-moore. I got lost in trying to debug the apparent netlify issue and missed your question in the comments.

I think what you have proposed sounds reasonable. I took a stab a cleaning things up a bit. Let me know what you think about the latest commit. Happy to revert if you have something else in mind.

manzt · 2021-08-10T22:34:44Z

src/io.ts

-    axis_labels = getAxisLabelsFromMultiscales(attrs);
+    if (!config.axis_labels) {
+      // Update config axis_labels if present in multiscales
+      config.axis_labels = attrs.multiscales[0].axes;


I think I'd rather just be explicit here for this edge case, rather than adding a util that might return undefined.

manzt · 2021-08-10T22:54:46Z

src/io.ts

@@ -24,18 +25,6 @@ import {
  RGB,
 } from './utils';

-function getAxisLabels(arr: ZarrArray, axis_labels?: string[], channel_axis?: number) {


Ah, I think channel_axis got lost when moving this to src/utils.ts. This is important for images without any dimension labels, as you can tell vizarr to separate out the channels in that dim like in napari.

will-moore · 2021-08-11T08:16:27Z

That looks great, thanks. Tested with the v0.3 links above and various v0.1 examples from the blog, as well as v0.3 data (without omero) https://oc.embl.de/index.php/s/KLTaSgX1c4zgnHP.

I also tried removing the omero data from multi-channel images, where one of the axes is labelled c.
Currently this is not considered a multi-channel image, and you get a c slider:

So I'll look at fixing that...

will-moore · 2021-08-11T09:23:11Z

That commit uses labels.indexOf('c') to pick channel_axis for non omero images, similar to the approach for napari: ome/napari-ome-zarr#9

manzt · 2021-08-12T20:34:37Z

Wonderful. I just tried it out! Going to go ahead and merge, thanks so much for working on this @will-moore.

cc: @joshmoore

will-moore mentioned this pull request Jul 12, 2021

v0.3 support #103

Closed

will-moore mentioned this pull request Jul 13, 2021

bump zarr with support for dimension_separator #105

Merged

will-moore mentioned this pull request Jul 14, 2021

export v0.3 ome ngff ome/omero-cli-zarr#70

Merged

will-moore force-pushed the ome_zarr_v0.3_tweaks branch from ed0106e to 5ca7b07 Compare July 15, 2021 07:28

manzt reviewed Jul 26, 2021

View reviewed changes

will-moore mentioned this pull request Aug 5, 2021

Add simple cache #100

Merged

will-moore added 8 commits August 5, 2021 16:54

Use multiscales.axes (v0.3) for axis_labels

5ebb3c3

loadOmeroMultiscales() handles v0.3 less than 5D

8fd8c69

Support v0.1 and v0.2 with default tczyx axes

26a3b08

format fixes

cf66227

Handle axes for OME Plate and Well

cc09826

getOmeAxisLabels(axes) takes single argument

e2029a2

Don't use LRUCacheStore(store)

3617cb8

hms-dbmi#100 (comment)

Move 'axes' logic to ome.ts

8ed9c89

will-moore force-pushed the ome_zarr_v0.3_tweaks branch from caaa02a to 8ed9c89 Compare August 5, 2021 16:49

joshmoore mentioned this pull request Aug 6, 2021

Update reader.py to v0.3 ome/ome-zarr-py#89

Merged

manzt mentioned this pull request Aug 6, 2021

Inspect netlify (?) bug #113

Closed

manzt force-pushed the ome_zarr_v0.3_tweaks branch from 067cb48 to 8ed9c89 Compare August 6, 2021 17:33

Non-omero images still read 'axes' from 'multiscales' attrs

452a63a

manzt reviewed Aug 10, 2021

View reviewed changes

manzt force-pushed the ome_zarr_v0.3_tweaks branch from 96a9271 to b299226 Compare August 10, 2021 22:47

manzt reviewed Aug 10, 2021

View reviewed changes

manzt force-pushed the ome_zarr_v0.3_tweaks branch from b299226 to f31a77e Compare August 10, 2021 23:08

Remove generic axes util

7de86be

manzt force-pushed the ome_zarr_v0.3_tweaks branch from f31a77e to 7de86be Compare August 10, 2021 23:09

Use labels.indexOf('c') to pick channel_axis

62464cc

manzt merged commit 0e02933 into hms-dbmi:main Aug 12, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ome zarr v0.3 tweaks #104

Ome zarr v0.3 tweaks #104

will-moore commented Jul 12, 2021 •

edited

Loading

joshmoore commented Jul 13, 2021

manzt commented Jul 13, 2021

manzt commented Jul 16, 2021

will-moore commented Jul 16, 2021

manzt commented Jul 17, 2021

manzt left a comment

manzt Jul 26, 2021

will-moore Aug 5, 2021

manzt commented Aug 5, 2021 •

edited

Loading

will-moore commented Aug 6, 2021

manzt commented Aug 6, 2021

will-moore commented Aug 9, 2021

manzt commented Aug 10, 2021

manzt Aug 10, 2021

manzt Aug 10, 2021

will-moore commented Aug 11, 2021

will-moore commented Aug 11, 2021

manzt commented Aug 12, 2021

Ome zarr v0.3 tweaks #104

Ome zarr v0.3 tweaks #104

Conversation

will-moore commented Jul 12, 2021 • edited Loading

joshmoore commented Jul 13, 2021

manzt commented Jul 13, 2021

manzt commented Jul 16, 2021

will-moore commented Jul 16, 2021

manzt commented Jul 17, 2021

manzt left a comment

Choose a reason for hiding this comment

manzt Jul 26, 2021

Choose a reason for hiding this comment

will-moore Aug 5, 2021

Choose a reason for hiding this comment

manzt commented Aug 5, 2021 • edited Loading

will-moore commented Aug 6, 2021

manzt commented Aug 6, 2021

will-moore commented Aug 9, 2021

manzt commented Aug 10, 2021

manzt Aug 10, 2021

Choose a reason for hiding this comment

manzt Aug 10, 2021

Choose a reason for hiding this comment

will-moore commented Aug 11, 2021

will-moore commented Aug 11, 2021

manzt commented Aug 12, 2021

will-moore commented Jul 12, 2021 •

edited

Loading

manzt commented Aug 5, 2021 •

edited

Loading