set-scale utility

[aaronalvarezcz]

2024-09-17 update from @talonchandler.

Problem: Scale metadata in existing .zarr stores is frequently incorrect.

Solution: A small CLI utility set-scale that can fix incorrect metadata in a single CLI call.

Previous work: the shrimPy repository included an earlier version of this utility..

Interface:

iohub set-scale -i input.zarr/ -z 2.0 -y 0.325 -x 0.325

will result in

Updating all.zarr/A/1/0 z scale from 2.3 to 2.0.
Updating all.zarr/A/1/0 y scale from 2.6 to 0.325.
Updating all.zarr/A/1/0 x scale from 2.6 to 0.325.
Updating all.zarr/A/1/1 z scale from 2.3 to 2.0.
...
...
...
Updating all.zarr/D/4/8 x scale from 2.6 to 0.325.

Other notes:

• I (@talonchandler) needed to move several parsing utilities into iohub. These utilities are widely used across compmicro's pipelines, so I think they deserve a place here.
• -i input.zarr will expand a plate into a list of positions, then update the scale metadata of each position. -i input.zarr/*/*/* is equivalent.
• this utility now accepts any subset of -t, -z, -y, and -x flags

[talonchandler]

I'll suggest:

• reviving the -i option
• trialing a full CLI and/or config interface instead of the current interactive
• documenting and testing

[ziw-liu]

Please add a description of the feature you are implementing and the problem it solves.

[talonchandler]

After discussion, here's a list of TODOs before we're ready for wider review:

• remove the dependency on mantis and copy over the -i option instead.
• move the test from shrimpy over to iohub
• make sure that tests are passing

[ieivanov]

It's great that the OptionEatAll is now moving it iohub. It'll be good to add tests for it, and also test that -i input.zarr gets expanded into positions.

[talonchandler]

@ieivanov I've added tests for OptionEatAll and plate-expansion behavior. I think this is ready for a final look.

[ieivanov]

I've run out of time debugging, but I managed to fix an issue with updating the scale transform when the transform did not exist in the first place (datasets are created with identity transform by default).

I'm still seeing an issue that in the .zattrs file only "old_x": 1.0 is written even when updating all of X, Y, and Z.

[talonchandler]

@ziw-liu our original line didn't update the metadata in the way we intended. My fault for not testing this.

@ziw-liu this is the simplest solution I could get to work, but it's a bit ugly. Should .zattrs["iohub"] be part of the pydantic models so that we can use dump_meta?

[ieivanov]

Let's merge!