Core algorithms for star trajectories analysis

[mexanick]

Please describe the use case that requires this feature.
The reconstruction of the star positions and analysis of their apparent shape and trajectories can be used for pointing and optical PSF monitoring. Assuming the known optical PSF, one can reconstruct the star trajectories and deduce the telescope pointing without using auxiliary hardware, and vice versa, assuming precise knowledge of pointing, one can reconstruct the PSF by identifying the cluster of pixels observing the light from a point-like source (a star).
The analysis of the star trajectories has been implemented in a standalone startracker package and the results of the corresponding analysis of the LST-1 data are accepted to publication by Astronomy & Astrophysics.

Describe the solution you'd like

I propose to port the algorithms and containers of the startracker code to the ctapipe and implement the orchestration tools in the calibration pipeline of the dpps. Currently, the implemented code contains the following modules:

• startracker.analysis.image: contains a set of containers and a StarExtractor component, providing the methods to reconstruct star positions. Proposal: implement containers in ctapipe.containers, implement StarExtractor in ctapipe.image.extractor. The output of the StarExtractor is a StarImageCollection container, similar to a DL1 container.
• startracker.analysis.fitter: contains two classes, providing methods to fit the star trajectories and extract the pointing correction. Proposal: implement in ctapipe.calib.pointing
• startracker.analysis.psf_model: contains a class, providing a PSF parametrization model. Proposal: implement in ctapipe.calib.psf

Additional context

Current class diagram, extracted from a standalone implementation. Subject to a heavy refactoring during the porting.

[maxnoe]

Hi @mexanick

Thanks for bringing this up.

The analysis of the star trajectories has been implemented in a standalone startracker package

This seems to be a private repository in the CERN gitlab and CERN does not seem to let me login into the Gitlab with my external account anymore, this used to work some time ago.

Could you move it somewhere where we can see it? E.g. the calibpipe group in the CTAO gitlab?

I propose to port the algorithms and containers of the startracker code to the ctapipe and implement the orchestration tools in the calibration pipeline of the dpps.

Given that you already have standalone package for a well-defined scope, what would be the advantages of integrating it into ctapipe itself compared to having the startracker module maintained by calibpipe and as a dependency of it?
Is there something you think can be done better by integrating it into ctapipe itself?

startracker.analysis.psf_model: contains a class, providing a PSF parametrization model. Proposal: implement in ctapipe.calib.psf

This sounds more like it should go into ctapipe.instrument.optics.

ctapipe.image.extractor. The output of the StarExtractor is a StarImageCollection container, similar to a DL1 container.

What does this extractor do? The job of the extractors in ctapipe.image is to compute the charge and peak time (what we call the dl1 image), it seems your star extractor would need to do something different (estimate pedestal variance from the waveform?).
If it is specific to this pointing calibration, I'd probably also have it in ctapipe.calib.pointing if we move towards integrating it with ctapipe.

[mexanick]

Hi @maxnoe, thank you for a quick feedback! I've made the repo public, you can browse it now.
I will not migrate it at the moment, until we decide whether it shall be ported to the ctapipe.

Considering the advantages of integrating it with the ctapipe:

• First of all, it relies heavily on ctapipe machinery for I/O and calibrations (well, some of these things are actually sourced from the lstchain, but the migration is foreseen). Thus, having it within the ctapipe will allow to integrate its own testing and ensure that breaking changes in ctapipe are correctly addressed.
• Secondly, the subsequent development foresees integration of the core startracker functionality for PSF monitoring (basically solving the reverse problem - if we know pointing, we can deduce the PSF).

This sounds more like it should go into ctapipe.instrument.optics.

This was actually my first choice, but I was a bit confused by the instrument module description, that says it holds configuration of the instrument. But if you prefer this, fine to me.

What does this extractor do? The job of the extractors in ctapipe.image is to compute the charge and peak time (what we call the dl1 image), it seems your star extractor would need to do something different (estimate pedestal variance from the waveform?).

Right, it extracts a variance image from waveform. It is indeed specific to calibration purposes (pointing and, eventually, PSF), so indeed, we can have it under ctapipe.calib, but probably more generic than pointing (could be astrometrics?)

[kosack]

This was actually my first choice, but I was a bit confused by the instrument module description, that says it holds configuration of the instrument. But if you prefer this, fine to me.

The instrument module is for static information about the instrument like the location of pixels and which camera is attached to which structure, not for things that change with time, which I would call calibration. Here "static" means rarely updated, or updated at about the same frequency as large hardware changes or simulation model upgrades (e.g. swapping of a camera). It is the implementation of the InstrumentDescription data model (at least up to the SubarrayDescription).

Currently the only "measured" quantity we include in the instrument module is the effective focal length. So the question is if the optical PSF is a similar quantity (something measured rarely when the instrument as a whole changes due to an upgrade), or something measured more often. Or is this more like the optical efficiency, which degrades in time, and should not be part of the instrument module?

[mexanick]

The PSF will be measured much more frequently than the focal length. Whether it will evolve over time is a different question... I can't answer it right now. The PSF model however should not change often, only its parameters. But indeed, it doesn't fit very well under "static" or "pseudo-static" instrument configuration.

[mexanick]

@kosack @maxnoe following your comments, can we agree on this implementation proposal?

• startracker.analysis.image: contains a set of containers and a StarExtractor component, providing the methods to reconstruct star positions. Proposal: implement containers in ctapipe.containers, implement StarExtractor in ctapipe.calib.astrometics (to accomodate subsequent use by pointing and psf monitoring tools). The output of the StarExtractor is a StarImageCollection container, similar to a DL1 container.
• startracker.analysis.fitter: contains two classes, providing methods to fit the star trajectories and extract the pointing correction. Proposal: implement in ctapipe.calib.pointing
• startracker.analysis.psf_model: contains a class, providing a PSF parametrization model. Proposal: implement in ctapipe.calib.psf (to accomodate subsequent use by pointing and psf monitoring tools).