Remove requirement for xarray-compatible metadata

[sbesson]

See #48 and ome/ome-zarr-py#166

With the introduction of axes, metadata at the multiscales dataset level allowing compatibility with xarray was added in the 0.3 version of the OME-NGFF specification. Some feedback and testing revealed the metadata currently required by the specification generates Zarr group which fail to open in xarray due to the different shapes in the resolution datasets.

This PR proposes a last-minute amendment to the 0.4 specification which is due for publication imminently to remove the xarray metadata. This should prevent libraries from writing datasets known to be not fit-for-purpose at the moment. In the mean time, the discussion can keep going on in #48 about the best way to ensure xarray-compatibility and lead to a formal proposal that can be released in an upcoming versions of the specification.

cc @thewtex

[constantinpape]

Good to go from my side.

[joshmoore]

👍 with the caveat that there's a desire to find a way forward with this:

• Store axes as _ARRAY_DIMENSIONS ome-zarr-py#166 (comment)
• Compatibility with xarray #48

[sbesson]

Summarizing the current state of the conversation happening in #48, it was confirmed the current specification is not expected to work with xarray without additional constraints.

For anyone wanting to use xarray with OME-Zarr, there are two working layouts:

• either use distinct axes names for the _ARRAY_DIMENSIONS of each multiscales datasets e.g. suffix it with the resolution index ["x0", "y0", "z0"] under 0/.zattrs, ["x1", "y1", "z1"] under 1/.zattrs ...
• or store the multiscale arrays under a nested layout - see https://github.com/ome/ngff/issues/48#issuecomment-1031668203`. In this case, the same axe names can be stored in the attributes of the dataset e.g. ["x", "y", "z"] under 0/<name>/.zattrs, ["x", "y", "z"] under 1/<name>/.zattrs ..

I'll leave this PR open until tomorrow morning for final vetoes/objections or suggestions and I am planning to get this in tomorrow morning to move forward with #86

[sbesson]

Merging as per #96 (comment).

@constantinpape how much more of #86, would you consider as a must have? Otherwise, I would be inclined to start going through the steps of https://github.com/ome/ngff#release-process and get the 0.4 version of the specification published.

[constantinpape]

@sbesson I think we can start the release process. The only "musts" left are making sure that it's possible to open the new spec in ome-zarr-py / vizarr / MoBIE. Implementation is pretty advanced for all three I think, so we don't need to block the release due to it. (But should maybe wait with advertising until it's possible to read it in the 3 tools.)

[sbesson]

👍 See #97. And totally agreed on waiting until we have a comprehensive set of samples/implementations before advertising. We also aim to publish a blog post on the OME-NGFF 0.4, I'll share the draft shortly