feat: move slow jumps and jumpfix to so3g #1081
Conversation
| if isinstance(jumps, np.ndarray): | ||
| jumps = RangesMatrix.from_mask(np.atleast_2d(jumps)) | ||
| elif isinstance(jumps, Ranges): | ||
| jumps = RangesMatrix.from_mask(np.atleast_2d(jumps.mask())) | ||
| if not isinstance(jumps, RangesMatrix): | ||
| raise TypeError("jumps not RangesMatrix or convertable to RangesMatrix") | ||
| jumps = cast(RangesMatrix, jumps) |
There was a problem hiding this comment.
This seems redundant to me, but it probably isn't. Why?
There was a problem hiding this comment.
Honestly that was just to make my typechecker shut up...
| @@ -310,6 +311,10 @@ def test_jumpfinder(self): | |||
| heights = heights[heights.nonzero()].ravel() | |||
| self.assertTrue(np.all(np.abs(np.array([10, -13, -8]) - np.round(heights)) < 3)) | |||
|
|
|||
| # Check fixing | |||
| ptp_fix = np.ptp(fixed.ravel()[~jumps.buffer(10).mask().ravel()]) | |||
| self.assertTrue(ptp_fix < 1.1*ptp_orig) | |||
There was a problem hiding this comment.
Is this really checking the jump fixing? It seems to me this is just checking that the fixed matrix is not messed up in the places where there were no jumps in the first place...
There was a problem hiding this comment.
Yeah so the jump fixing assumes everything within the jump range is the jump so you end up with an offset still inside the jump range (which we ignore since we gapfill those ranges anyways).
The ptp checks if we fixed because after the jump there is a DC offset that will be gone in the fixed TOD.
See below for an example (blue is before fixing, orange after):
If we wanted to not have the errors within the jump ranges in the fixed TOD you just need to lower the buffer size for the ranges matrix (but then your jump could be not in the range). The current range is set by the window size used for jump finding to give a reasonable margin of error.
There was a problem hiding this comment.
Right, thanks for the visual explanation, the ptp matter is clear now. Just for my understanding: is jumps.buffer(10).mask() flagging the jump regions plus a buffer region covering the 10 samples before and the 10 samples after the respective jump region?
There was a problem hiding this comment.
Yup, upped the buffer in the test to be conservative so that things never fail in CI. But also I set the PRNG key so I think that is unneeded.
|
simonsobs/so3g#196 is now merged so this should be good to go very soon |
sotodlib/tod_ops/jumps.py
Outdated
| from scipy.sparse import csr_array | ||
| from skimage.restoration import denoise_tv_chambolle | ||
| from so3g import ( | ||
| matched_jumps, | ||
| matched_jumps64, | ||
| block_minmax, |
There was a problem hiding this comment.
One way to create some backwards-compatibility is to not import these by name. Just import so3g. Then the code will crash when you try to use those functions, rather than when you try to import them.
As written now, everyone's installation will break, if they try to use anything from sotodlib.tod_ops, unless they've updated so3g.
skhrg
force-pushed
the
more_jump_speedups
branch
from
7aa611f to
c6dc69c
Compare
Relies on simonsobs/so3g#196
This is a direct rewrite for the most part, no algorithmic changes.