Misclassified cells in corners - plateU7rep1_20200519_210009_665 - C04-0005

[imagirom]

{"plateName":"plateU7rep1_20200519_210009_665","siteName":"C04-0005","pixelLocation":[11375.74293134145,7246.784387298996,0.0],"analysisVersion":"?"}

In many Images of the new plates, the infected/control classification fail in the corners, especially (at leas from my impression) the top left. A lot of control cells are classified as infected.

I think I know the reason: The Flatfield correction, which is more extreme in the new plates, enhances the noise in the corners, up to a factor in the order of 10. Hence, the 0.95 quantile, which is used as the feature for the cell classification, rises as well, leading to the misclassification.

Any Ideas for an easy fix?

Screenshot

[imagirom]

@tischi @constantinpape

[tischi]

@imagirom

1. Could you please also post here an image where one can see the noise?
2. If the noise is spatially uncorrelated a morphological opening with a radius of 1 pixel may remove the bright noise and only leave the dark noise. In truly infected cells the bright pixels are spatially connected and would thus survive the opening. Thus this may help. I actually thought we do such a morphological opening already anyway?

[imagirom]

@tischi
for now re 2.: We currently use the white tophat transform https://scikit-image.org/docs/0.12.x/auto_examples/xx_applications/plot_morphology.html#white-tophat

[tischi]

re (2): Maybe doing an additional radius=1 opening after that would help to remove spatial high frequency noise, depends on (1).

[imagirom]

This is before the white tophat (the scale is probably not ideal). The corner is in the top left.

And after the white tophat:

[tischi]

You can try a few things:

1. Do a 1 or 2 pixel median filter before you do the white top hat
2. Do a 1 or 2 pixel opening before you do the white top hat
3. Replace white top hat by subtracting a median filtered version of the image from itself. For the median filter you should use a similar radius as you did for the white top hat, maybe the radius even a little bit larger. Note: Here you can get negative values, so choose an appropriate datatype.

The general issue with a white top hat is that it does not work well with images that have a lot of noise. So either you try to remove the noise before applying it (1 and 2) or you don't use it, but rather do median subtraction (3).

[tischi]

@imagirom

...I am not so sure anymore if it makes a difference whether you remove the noise before or after the tophat. I have to play a bit with it myself.

Could I have access to the images just after flat-field correction (without any further processing)?
For example, are they accessible in the PlateViewer?

[imagirom]

@tischi Yes, the 'marker' channel is what you are looking for. After the tophat is called 'marker_tophat'. That minus background is 'marker_tophat_background_subtracted'. Before flat-field correction is 'marker_raw'.

[tischi]

And in which folder should I look on the kreshuk server?

[constantinpape]

/g/kreshuk/data/covid/data-processed

[tischi]

flat-field is very good

a bit higher noise in the corners...

...as expected because there we have less signal to the the illumination profile

tophat filter increases overall brightness of high noise regions

...this is an annoying property of the tophat = im - open(im). If you view this as im - background( im ), the opening operation underestimates the background in high noise regions, thus we subtract too little, thus the result is too bright.

[tischi]

The flatfield correction is so good now that I wonder wether we should just not do anything more. For sure the white top hat is problematic due to the high and spatially varying noise levels.

As we discussed some time ago, what I may have done is a connected component analysis on the pixels that are above threshold in each cell and only keep connected components that have a minimal size and then base the decision whether to classify a cell as infected based on that. That would be a way to distinguish noise from real virus regions.

As a quick fix I would suggest to try to simply remove the white top hat filter as it seems to do more harm than good.