Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add missing info about peak_time range #2563

Open
kosack opened this issue Jun 11, 2024 · 3 comments
Open

add missing info about peak_time range #2563

kosack opened this issue Jun 11, 2024 · 3 comments

Comments

@kosack
Copy link
Contributor

kosack commented Jun 11, 2024

Please describe the use case that requires this feature.

We are missing information to find the expected range of the peak_time output from an ImageExtractor when the CameraCalibrator.apply_peak_time_shift option is enabled. Since this shifts the peak times by some amount, converting from the peak_time in ns back to sample number, or predicting the range of possible peak times a priori is not possible
(without reading many events and looking for the min/max)

For example for a given telescope type, the width of the readout window is:

ttype = subarray.telescope_types[0]
readout_width = (ttype.camera.readout.n_samples/ttype.camera.readout.sampling_rate).to("ns")
# 100 ns

however, while we know the readout-window width, we do not know the starting point and this can't be reconstructed from the from the SubarrayDescription alone.

Some reasons one might want to do this:

  • make a histogram of peak time to look e.g. for edge effects where you don't know a-priori the starting and ending peak time values, in a generic way that works for any camera.
  • to debug camera to be able to go from peak_time in ns back to a peak_time in readout samples number units.

Describe the solution you'd like

  1. store the time-shift calibration information in the monitoring output like other calibration coefficients so that it can be "undone" if needed
  2. store the peak time in both ns and samples (costly for file size)
  3. only do this kind of debugging/benchmarking with files whereapply_peak_time_shift and apply_waveform_shift are False (not ideal, since they are on by default, and since they affect the performance of the time reconstruction)
@maxnoe
Copy link
Member

maxnoe commented Jun 11, 2024

Is the issue here pre-defining the range of histogram edges?

While it doesn't address the underlying issue, doing it exactly right is also not something practical, since for real data, you will have many flat fielding time offsets and not just the single one. So you need to do a global min/max over all monitoring information to define the range.

How about just defining a safe range, like -10 ns / window_width + 10 ns and then don't display empty bins at the edges.

@kosack
Copy link
Contributor Author

kosack commented Jun 12, 2024

e, doing it exactly right is also not something practical, since for real data, you will have many flat fielding time offsets and not just the single one. So you need to do a global min/max over all monitoring information to define the range.

yes, currently we are doing essentially that (a larger range, since the edges will be "fuzzy" due to the different time offsets). It's not possible to test for edge effects, but that may just be something we can't do, or just need to re-generate the DL1 with time-offsets disabled to test it.

But having a way to debug/identify issues with the readout or the extraction algorithm may also be useful in the future for observed data as well: I was thinking also of data quality monitoring where you might want to check that the waveforms have no "hotspots" (like in the current LST1 data where a few bins are high). But that could be done at the DL0 level, and not using peak_time.

I think in any case we should have a way to get back the R1 calibration coefficients, which include the temporal flat-fields as well as spatial FF, peds, etc. For simulations, this could just be propegated to the DL1 files

@kosack
Copy link
Contributor Author

kosack commented Jun 12, 2024

For context: we were trying to debug a strange feature in the reconstructed charge distribution that wasn't present (or minimally present) in point-like prod3b sims, but appears in prod5 and above. It turned out to be due to two effects: pulses clipped by the readout window (which happens more often in diffuse simulations), and afterpulses that end up in the readout window following a real but clipped pulse. So in the end these were real effects, and not issues with the extractor, but it was a bit hard to get to that point with DL1 data only and without absolute times in samples.

In the end, it's not a huge issue, as we can require that for the DL1 benchmarks we also include some DL0 waveforms in the data and/or disable temporal flat fields.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants