zero-intensity mask in correlations

[tjlane]

@dermen @CoChrists

Gundolf pointed out that in the old c implementation of the correlation function, zeros were used as a mask proxy. Is this OK? Should we change it?

https://github.com/tjlane/thor/blob/master/src/python/xray.py#L3395
https://github.com/tjlane/thor/blob/master/src/corr/corr.cpp#L49

We should at least document this.

@CoChrists are there other places in the code where this happens that I missed?

[CoChrists]

In my opinion this should be changed. A zero intensity is a valid value. How about filtering by negative values, i.e. multiplying the (no-neg.) intensity values of the masked pixels by -1?

Basically all places where a mask is set up or applied have to be checked...

[dermen]

Hmm, in my stand-alone code I use a user defined value to represent the mask flag. In fact , the thor stand alone c++ correlation code is just a copy of my stand alone c code, which I have since updated:

https://github.com/dermen/popi/blob/master/src/corr/corr.cpp

In my data blocks I represent masked pixels as -1, so when I call popi.correlate I let mask_val_ = -1. If I am doing difference intensities I let mask_val_ = 0 because -1 - (-1) = 0. The error associated with accidentally masking pairs of pixels who both have intensity values = 0 is vanishingly small in the case of total scattering measurements.

But @CoChrists , in the main correlation implementation in thor ( Rings.correlate_intra), a mask is provided by the user that consists of 0 and 1 values. It doesnt care if there is a zero intensity; it only cares if a zero is supplied by the user in the mask object (at least that's how it should be functioning).

For example, say I have the pixels

phi0 = 11
phi1 = 10
phi2 = 0
phi3 = 13
phi4 = 12
phi5 = 0

and I want to mask all even pixels. Then I would make a mask that looks like

mask_phi0 = 0
mask_phi1 = 1
mask_phi2 = 0
mask_phi3 = 1
mask_phi4 = 0
mask_phi5 = 1

and then thor.xray.Rings.correlate_intra would only consider the pixels

phi1 = 10
phi3 = 13
phi5 = 0

in the computation. So I do not know how necessary it is to make adjustments, other than an adjustment to thor.corr.correlate (which has alias brute_correlate in xray.py), which we can simply replace by the code in the above link

[tjlane]

thanks @dermen -- do you think it's worth updating to your most recent version? sounds like you should be able to open a PR and get it in w/o any work.

@CoChrists are you happy with this? if not speak up :)