ongoing efforts

[tcompa]

• Moved tasks to API “V2” => 3 types (0,1,N output images) to 2 types (0+N or 1).

• Unified handling of no-image-input and multi-image-input

• Basic examples for new_zarr+MIP and new_zarr+copy_data

• Important warning: N-dimensional parallelization (vs one-dimensional parallelization across images) is immediately risky re: huge number of parallelization items. [...]

• New example: custom init task (illumination correction parallelized over channels)

• New example: custom init task for registration, which pairs a reference cycle with all other cycles of each well. Note: well-handling is part of the task.

• Formalized a first version of the task-output API, via TaskOutput model

• Deduplicate redundant new-images list coming from subset parallelization.

• Example of root_dir vs absolute path - ref Absolute paths #2

• Source-to-output mapping: Propagating image attributes in parallel tasks (without the task needing to handle it)

• Remove default values for filters, and only consider those that are set explicitly. EDIT This is currently covered, by defaulting to None. A further review is needed.

• TO CONSIDER: using both include&exclude filters, e.g. to enable selecting "images that do not have a certain attribute equal to something". EDIT: this is currently postponed, as the main intended use case (3D/2D/3D) is nicely covered by the dim=3,dim=2,dim=3 pattern (that is, using a string flag rather than a boolean attribute)

• Review and clean up the parallel-task-output merge strategies

• Add safety check: Total number of custom parallel items is limited (we can later enable a safe=False override)

• Add safety check: no unintended use case is reached when applying both filters and parallelization, if there was an init task. Let’s start with very explicit if/else and error messages, and then iterate.

• Fully enable 3D->2D->3D workflow example.

---

• TODO: take decision on root_dir vs absolute paths. At the moment none of the two options is blocking. The absolute-paths option requires a bit more string parsing, but that's OK (and part of it can be easily extracted into utils functions, like

def extract_base_parts_from_list_of_absolute_paths(absolute_image_paths: listr[str]) -> dict[str, Any]:
    ...
    return dict(absolute_root_dir=absolute_root_dir, plate_names=plate_names)

• Review strategy for backwards-compatibility

[jluethi]

Discussion on "Remove default values for filters, and only consider those that are set explicitly":
(path = "plate.zarr/B/03/0")
(path = "plate.zarr/B/03/0_corr", raw=False)

Still have ability to load raw=False and only return the second item

[jluethi]

Some validation of tasks would be very interesting. For example:
MIP requires images with "Dimensionality = 3D"

[tcompa]

To evaluate:
Should parallel tasks return something like new_image and/or edited_image, rather than new_images and/or edited_images? TBD

[tcompa]

To evaluate:
How can an image be removed from dataset.images?
First proposal: tasks don't remove images, but the fractal-server API makes it easy to play with dataset.images if needed
Second proposal: next to new_images and edited_images, a task can also output removed_images.

TBD