Basic image segmentation post (with dask-image)

[GenevieveBuckley]

Outline for a blogpost on a basic image segmentation pipeline using dask-image. See #47 (comment)

Content from github.com/GenevieveBuckley/dask-image-talk-2020

[GenevieveBuckley]

"Eagle eyed readers will notice that not every nucleus is separated perfectly from its neighbours" (& add link to Nick or Juan's napari interactive segmentation tutorial?)

[GenevieveBuckley]

Would be good to have a section on piping the output into a Dask dataframe directly

[GenevieveBuckley]

This has been up for long enough that people have had a fair chance to leave comments, so I'm going to go ahead and merge it now.

[GenevieveBuckley]

I did think about whether this should have been a runnable notebook in the dask-examples, but I don't think we can stay under the suggested 2GB memory limitation for it.

[jakirkham]

Sorry didn't know this was here 😅

Anyways this looks great! Thanks for doing this 😄

[GenevieveBuckley]

Sorry didn't know this was here sweat_smile

That's probably my fault @jakirkham, I'll try to remember to ping you by name on future posts in your area of interest.

Anyways this looks great! Thanks for doing this smile

Thanks! It's clunky in two areas:

1. At the end of the pipeline, it would be better to put the results into a dask dataframe (one row per nucleus, with measurements of area & intensity). I wasn't sure how to do that well, but I also didn't spend much time trying. Adding this to examples would be useful.
2. No watershed in dask-image, which is an issue even in this small example. I know that's a technically difficult problem (Added a port of scipy.ndimage.measurement.watershed. dask-image#99 is stalled)

[jakirkham]

Not at all. It's been a pretty busy month anyways

Idk what parts were tricky with converting to Dask Dataframes, but maybe this simple example helps? If not, would be curious to know where the issues are

In [1]: import numpy as np
   ...: import pandas as pd
   ...: import dask
   ...: import dask.array as da
   ...: import dask.dataframe as dd

In [2]: a = da.stack(3 * [da.arange(1_000, chunks=100)])
   ...: a
Out[2]: dask.array<stack, shape=(3, 1000), dtype=int64, chunksize=(1, 100), chunktype=numpy.ndarray>

In [3]: d = a.T.to_dask_dataframe(["a", "b", "c"])
   ...: d
Out[3]: 
Dask DataFrame Structure:
                    a      b      c
npartitions=10                     
0               int64  int64  int64
100               ...    ...    ...
...               ...    ...    ...
900               ...    ...    ...
999               ...    ...    ...
Dask Name: from-dask, 90 tasks

Yeah that's true remembering talking to Juan about this after we finished connected components. It's tricky, but may be doable