Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lift 5d requirement for images and move multiscales description into spec #39

Closed
wants to merge 4 commits into from
Closed

Lift 5d requirement for images and move multiscales description into spec #39

Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
e1a5168
Lift 5d requirement for images
constantinpape Mar 13, 2021
9fae845
Move multiscales definition into spec
constantinpape Mar 13, 2021
4223549
Address reviews: make axes MUST; bump version to 0.2
constantinpape Mar 16, 2021
7b664a9
Address review comments
constantinpape Mar 17, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 48 additions & 4 deletions latest/index.bs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -124,8 +124,8 @@ multiple levels of resolutions and optionally associated labels.
├── n # The name of the array is arbitrary with the ordering defined by
│ │ # by the "multiscales" metadata, but is often a sequence starting at 0.
│ │
│ ├── .zarray # All image arrays are 5-dimensional
│ │ # with dimension order (t, c, z, y, x).
│ ├── .zarray # All image arrays are up to 5-dimensional with dimension order (t, c, z, y, x).
│ │ # An image without time dimension has axes (c, z, y, x) etc.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is a 3D array interpreted as (z, y, x)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we promote "axes" to MUST, then the interpretation always depends on the metadata, i.e. on the values in "axes".

│ │
│ ├── 0.0.0.0.0 # Chunks are stored with the flat directory layout.
│ │ ... # Each dotted component of the chunk file represents
Expand Down Expand Up @@ -207,8 +207,52 @@ keys as specified below for discovering certain types of data, especially images

Metadata about the multiple resolution representations of the image can be
found under the "multiscales" key in the group-level metadata.
The specification for the multiscale (i.e. "resolution") metadata is provided
in [zarr-specs#50](https://github.com/zarr-developers/zarr-specs/issues/50).
"multiscales" contains a list of dictionaries where each entry describes a multiscale image.

"multiscales" MUST contain the field "datasets", which is a list of dictionaries describing
the arrays storing the individual resolution levels. Each dictionary in "datasets" MUST
contain the field "path", whose value contains the path to the array for this resolution relative
to the root zarr group. The "path"s MUST be ordered from largest (i.e. highest resolution) to smallest.

It SHOULD contain the field "name".

It SHOULD contain the field "version", which indicates the version of the
multiscale metadata of this image (current version is 0.1).

It SHOULD contain the field "axes", which is a list of dimension names of the axes.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the default behavior if axes is not specified? In particular should the consumer have an expectation with regards to the number of underlying dimensions or try to detect it?

Alternatively, the lifting of the 5D requirement could lead to increasing the requirement for axes at a MUST level.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, maybe it's better to change axes to MUST. I went for SHOULD to stay compatible with 0.1; but if hierarchical storage becomes MUST (see #29) 0.1 will be deprecated anyway and we can also promote axes to MUST.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any thoughts @joshmoore?

Copy link
Member

@joshmoore joshmoore Mar 16, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No immediate objection to breaking now along with this loosening of the 5D/strict-ordering version, but it might be nice as a part of this effort to plan out the next 1 or 2 large changes based on #35 and see which are also going to be breaking.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No immediate objection to breaking now along with this loosening of the 5D/strict-ordering version, but it might be nice as a part of this effort to plan out the next 1 or 2 large changes based on #35 and see which are also going to be breaking.

Good point. I will try to summarise the current discussion in there later so we can see what else down the line might be breaking.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed to MUST now, also summarised potential changes: #35 (comment). As far as I can see none of this would be breaking with the current proposal for 0.2.

If given, the values MUST be unique and one of `{"t", "c", "z", "y", "x"}`.
The number of values MUST be the same as the number of dimensions of the arrays corresponding to this image.

It SHOULD contain the field "type", which gives the type of downscaling method used to generate the multiscale image pyramid.
It SHOULD contain the field "metadata", which contains a dictionary with additional information about the downscaling method.

```json
{
"multiscales": [
{
"version": "0.1",
"name": "example",
"datasets": [
{"path": "0"},
{"path": "1"},
{"path": "2"}
],
"axes": [
"t", "c", "z", "y", "z"
],
"type": "gaussian",
"metadata": { # the fields in metadata depend on the downscaling implementation
"method": "skimage.transform.pyramid_gaussian", # here, the paramters passed to the skimage function are given
"version": "0.16.1",
"args": "[true]",
"kwargs": {"multichannel": true}
}
}
]
}
```


If only one multiscale is provided, use it. Otherwise, the user can choose by
name, using the first multiscale as a fallback:

Expand Down