Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Coordinate transforms json-schema (request for feedback) #149

Draft
wants to merge 56 commits into
base: main
Choose a base branch
from

Conversation

ivirshup
Copy link
Contributor

@ivirshup ivirshup commented Oct 10, 2022

json-schema for coordinate transforms proposal #138.

This branch is based off the spec PR. The spec was only modified to move inline examples to the examples directory. All other changes should be to files under:

latest/examples
latest/schemas
latest/tests

This isn't 100% done, but I would appreciate feedback on the implementation and structure of the schemas.

The plan

@bogovicj and I would like to get feedback on the schema here. At a later point we would like to merge it with the coordinate transform spec.

What's left to do:

Questions from me

  • As structured, this is not extensible. Transformations are explicitly allowed. Suggestions for how to make this extensible, while not being too permissive are welcome.
  • How could I test that arrays and multiscales must have coordinate transform metadata stored on them (See spec: 027b55b)? All current example just test on stand-alone metadata documents. It would be good to be able to validate an entire store. How could this be achieved? The consolidated metadata representation of a store?

Current state of tests

Since the CI isn't currently running, I figured I'd show the state of the tests here:

pytest output
$ pytest -v --tb=no
===================================================== test session starts =====================================================
platform darwin -- Python 3.9.12, pytest-7.1.2, pluggy-1.0.0 -- /usr/local/opt/[email protected]/bin/python3.9
cachedir: .pytest_cache
rootdir: /Users/isaac/github/ngff/latest
plugins: xdist-2.5.0, assume-2.4.3, nunit-1.0.0, anyio-3.4.0, profiling-1.7.0, forked-1.3.0, cov-3.0.0
collected 59 items                                                                                                            

tests/test_validation.py::test_run[validate_multiscales_example] PASSED                                                 [  1%]
tests/test_validation.py::test_run[validate_multiscales_transformations] PASSED                                         [  3%]
tests/test_validation.py::test_run[validate_image_metadata] PASSED                                                      [  5%]
tests/test_validation.py::test_run[validate_image] PASSED                                                               [  6%]
tests/test_validation.py::test_run[validate_image_omero] PASSED                                                         [  8%]
tests/test_validation.py::test_run[validate_mismatch_axes_units] PASSED                                                 [ 10%]
tests/test_validation.py::test_run[validate_untyped_axes] PASSED                                                        [ 11%]
tests/test_validation.py::test_run[validate_missing_version] PASSED                                                     [ 13%]
tests/test_validation.py::test_run[validate_invalid_axis_units] PASSED                                                  [ 15%]
tests/test_validation.py::test_run[validate_missing_name] PASSED                                                        [ 16%]
tests/test_validation.py::test_run[validate_custom_type_axes] PASSED                                                    [ 18%]
tests/test_validation.py::test_run[validate_duplicate_axes] PASSED                                                      [ 20%]
tests/test_validation.py::test_run[validate_missing_space_axes] PASSED                                                  [ 22%]
tests/test_validation.py::test_run[validate_invalid_transformation_type] PASSED                                         [ 23%]
tests/test_validation.py::test_run[validate_missing_scale] PASSED                                                       [ 25%]
tests/test_validation.py::test_run[validate_too_many_axes] PASSED                                                       [ 27%]
tests/test_validation.py::test_run[validate_invalid_channels_color] PASSED                                              [ 28%]
tests/test_validation.py::test_run[validate_missing_axes_name] PASSED                                                   [ 30%]
tests/test_validation.py::test_run[validate_invalid_axes_count] PASSED                                                  [ 32%]
tests/test_validation.py::test_run[validate_one_space_axes] PASSED                                                      [ 33%]
tests/test_validation.py::test_run[validate_invalid_path] PASSED                                                        [ 35%]
tests/test_validation.py::test_run[validate_invalid_multiscales_transformations] PASSED                                 [ 37%]
tests/test_validation.py::test_run[validate_missing_transformations] PASSED                                             [ 38%]
tests/test_validation.py::test_run[validate_no_datasets] PASSED                                                         [ 40%]
tests/test_validation.py::test_run[validate_missing_datasets] PASSED                                                    [ 42%]
tests/test_validation.py::test_run[validate_missing_axes] PASSED                                                        [ 44%]
tests/test_validation.py::test_run[validate_invalid_version] PASSED                                                     [ 45%]
tests/test_validation.py::test_run[validate_invalid_axis_type] PASSED                                                   [ 47%]
tests/test_validation.py::test_run[validate_duplicate_scale] PASSED                                                     [ 49%]
tests/test_validation.py::test_run[validate_no_axes] PASSED                                                             [ 50%]
tests/test_validation.py::test_run[validate_too_many_space_axes] PASSED                                                 [ 52%]
tests/test_validation.py::test_run[validate_no_multiscales] PASSED                                                      [ 54%]
tests/test_validation.py::test_run[validate_invalid_channels_window] PASSED                                             [ 55%]
tests/test_validation.py::test_run[validate_empty_transformations] PASSED                                               [ 57%]
tests/test_validation.py::test_run[validate_missing_path] PASSED                                                        [ 59%]
tests/test_validation.py::test_run[example_multiscales_example] FAILED                                                  [ 61%]
tests/test_validation.py::test_run[example_multiscales_transformations] PASSED                                          [ 62%]
tests/test_validation.py::test_run[example_multiscales_example_relative] FAILED                                         [ 64%]
tests/test_validation.py::test_run[example_byDimension2] PASSED                                                         [ 66%]
tests/test_validation.py::test_run[example_mapIndex2] PASSED                                                            [ 67%]
tests/test_validation.py::test_run[example_scale] PASSED                                                                [ 69%]
tests/test_validation.py::test_run[example_byDimensionInvalid2] PASSED                                                  [ 71%]
tests/test_validation.py::test_run[example_mapAxis1] PASSED                                                             [ 72%]
tests/test_validation.py::test_run[example_coordinates1d] FAILED                                                        [ 74%]
tests/test_validation.py::test_run[example_affine2d2d] PASSED                                                           [ 76%]
tests/test_validation.py::test_run[example_affine2d3d] PASSED                                                           [ 77%]
tests/test_validation.py::test_run[example_xarrayLike] FAILED                                                           [ 79%]
tests/test_validation.py::test_run[example_rotation] PASSED                                                             [ 81%]
tests/test_validation.py::test_run[example_identity] PASSED                                                             [ 83%]
tests/test_validation.py::test_run[example_translation] PASSED                                                          [ 84%]
tests/test_validation.py::test_run[example_displacement1d] FAILED                                                       [ 86%]
tests/test_validation.py::test_run[example_sequenceSubspace1] FAILED                                                    [ 88%]
tests/test_validation.py::test_run[example_mapAxis2] PASSED                                                             [ 89%]
tests/test_validation.py::test_run[example_sequence] PASSED                                                             [ 91%]
tests/test_validation.py::test_run[example_inverseOf] FAILED                                                            [ 93%]
tests/test_validation.py::test_run[example_byDimensionInvalid1] PASSED                                                  [ 94%]
tests/test_validation.py::test_run[example_byDimension1] PASSED                                                         [ 96%]
tests/test_validation.py::test_run[example_mapIndex1] PASSED                                                            [ 98%]
tests/test_validation.py::test_run[example_arrayCoordSys] PASSED                                                        [100%]

=================================================== short test summary info ===================================================
FAILED tests/test_validation.py::test_run[example_multiscales_example] - jsonschema.exceptions.ValidationError: 'axes' is a ...
FAILED tests/test_validation.py::test_run[example_multiscales_example_relative] - jsonschema.exceptions.ValidationError: 'co...
FAILED tests/test_validation.py::test_run[example_coordinates1d] - jsonschema.exceptions.ValidationError: {'name': 'a coordi...
FAILED tests/test_validation.py::test_run[example_xarrayLike] - jsonschema.exceptions.ValidationError: {'type': 'byDimension...
FAILED tests/test_validation.py::test_run[example_displacement1d] - jsonschema.exceptions.ValidationError: {'name': 'a displ...
FAILED tests/test_validation.py::test_run[example_sequenceSubspace1] - jsonschema.exceptions.ValidationError: {'type': 'sequ...
FAILED tests/test_validation.py::test_run[example_inverseOf] - jsonschema.exceptions.ValidationError: {'type': 'inverseOf', ...
================================================ 7 failed, 52 passed in 0.24s =================================================

bogovicj and others added 30 commits February 1, 2022 13:50
* define pixel center coordinate system
* add input/output_axes
* add transform inverse flag
* add affine transform type
* flesh out array space
* define array indexing
* add more transformation types
* start transformation details section and examples
* update example
* use "input" and "output" rather than '*Space' and '*Axes'
* reorder details
* clean up table
* add rotation
* details for sequence
* describe inverses
* wrap examples
* rephrase matrix storage
* change to "coordinates", removing "Field"
* change to "displacements", removing "Field"
* add details for transformation types
* (identity, inverseOf, bijection)
* describe inputAxes and outputAxes
* add new examples
* add mapIndex, mapAXis
* add examples
* affine stored as flat array only
* sequence does not have by-dimension behavior
* flesh out some examples
@github-actions
Copy link
Contributor

Automated Review URLs

"type": "scale",
"scale": [0.1, 1.0, 0.5, 0.5, 0.5],
"input" : "/0",
"output" : "xampleCoordinateSystem"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exampleCoordinateSystem ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -0,0 +1,64 @@
{
"coordinateSystems" : [
{ "name " : "in", "axes" : [ {"name" : "0", "name" : "1", "name": "2", "name": "3", "name": "4" }] },
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The axes list has a single dict with multiple "name" keys? Is is supposed to be [{"name" : "0"}, {"name" : "1"}..] ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
{ "name " : "in", "axes" : [ {"name" : "0", "name" : "1", "name": "2", "name": "3", "name": "4" }] },
{ "name ": "in", "axes": [{"name" : "0"}, {"name" : "1"}, {"name": "2"}, {"name": "3"}, {"name": "4"}] },

@bogovicj, should changes to the examples happen here or in your PR?

I haven't modified these from your branch, but also could add tests for the subspace directory.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch, I'll fix

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

{
"coordinateSystems" : [
{ "name " : "in", "axes" : [ {"name" : "0", "name" : "1", "name": "2", "name": "3", "name": "4" }] },
{ "name " : "out", "axes" : [ {"name" : "x", "name" : "y", "name" : "z" }] }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch, I'll fix

"coordinateTransformations" : [
{
"type" : "sequence",
"name" : "5D-to-3D",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this combination of mapIndex and scale really go from 5D data to 3D? And same for 5D-to-3D-not-contiguous examples below?

"time",
"channel",
"coordinate",
"displacement"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The spec proposal says It SHOULD be one of "array", "space", "time", "channel", "coordinate", or "displacement" but MAY take other values for custom axis types that are not part of this specification yet but this schema wouldn't allow any other custom types.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How should schemas for this repository represent "MAY"?

Is this what's supposed to be in the strict_* schemas?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The strict schemas are designed to give warnings that encourage more complete metadata (e.g. multiscales SHOULD have a "name"). If someone has used some axes "type": "my_custom_type" there is probably a good reason (and it maybe shouldn't fail a strict schema validation). However the spec does say It SHOULD be one of "array", "space", "time"... and the strict schemas are used for SHOULD rules, so that is probably the right place to add that (rather than not include it at all).

Hopefully it's not too tricky to move it there?

}]
}
],
"coordinateTransformations": [{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the top-level multiscales coordinateTransformations no-longer valid? If it is, then this is a useful example. If not then the spec wording needs to be changed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will still be valid - I'll edit this example showing how.

"datasets": [
{
"path": "0"
// the transformation of other arrays are defined relative to this, the highest resolution, array
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The spec still says:

"Each dictionary in "datasets" MUST contain the field "coordinateTransformations", which contains a list of transformations that map the data coordinates to the physical coordinates (as specified by "axes") for this resolution level.
The transformations are defined according to [[#trafo-md]]. The transformation MUST only be of type translation or scale.
They MUST contain exactly one scale transformation that specifies the pixel size in physical units or time duration. If scaling information is not available or applicable for one of the axes, the value MUST express the scaling factor between the current resolution and the first resolution for the given axis, defaulting to 1.0 if there is no downsampling along the axis."

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bogovicj, assuming I should go with the spec over the example here?

Out of interest, why does the pixel -> physical need to be replicated here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the spec just needs updating or clarifying at that point. Maybe now it's

"Each dictionary in "datasets" except the first MUST contain the field "coordinateTransformations"...

bogovicj and others added 2 commits October 12, 2022 13:39
@bogovicj
Copy link
Contributor

bogovicj commented Dec 7, 2023

@ivirshup, after a long break and silence, I looked at this PR, and it's been a huge help in updating the schema!
Thank you so much.

Soon (today?) I'm going to merge this into my branch, and your commits will be merged when my PR is merged.

I'll also try to address some of the issues caught bu @will-moore, thanks for catching them!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants