Raw data should be quantized

[hugobuddel]

Currently the raw data is now float64, but it should be int16 when NDIT==1, and float32 when NDIT>1.

The Quantization Effect can be used for this.

See the data rates document.

The data rates document does not seem to take this doubling of the data size into account when NDIT>1; I'll take that up with Roy in the context of AstarVienna/METIS_DRLD#214 .

I think we should limit ourselves now to NDIT==1 and do NDIT>1 once we get there.

[astronomyk]

Please add as a follow up to the disccussion worh roy that in the N-band a 1s DIT will probably already saturate the detector if we use 16-bit ints. How exactly we deal with that is not clear to me. A quick check with how-many-bloody-photons for the expected background levels for L, M, N bands should show whether this is also a problem for M band ... I'm unsure here too. Hugo Buddelmeijer ***@***.***> schrieb am Do., 29. Feb. 2024, 10:41:

…

Currently the raw data is now float64, but it should be int16 when NDIT==1, and float32 when NDIT>1. The Quantization Effect can be used for this. See the data rates document <https://docs.google.com/spreadsheets/d/12krmTLojs9rRH6wuXFQFeUUBCde7G1_JwOGwDVd8Mwo/edit#gid=687079197> . The data rates document does not seem to take this doubling of the data size into account when NDIT>1; I'll take that up with Roy in the context of AstarVienna/METIS_DRLD#214 <AstarVienna/METIS_DRLD#214> . I think we should limit ourselves now to NDIT==1 and do NDIT>1 once we get there. — Reply to this email directly, view it on GitHub <#5>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACH66QTSTH2GN77CTMBTRH3YV33V3AVCNFSM6AAAAABD7SWFG2VHI2DSMVQWIX3LMV43ASLTON2WKOZSGE3DAOBSHA4DGMY> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[hugobuddel]

Please add as a follow up to the disccussion worh roy that in the N-band a 1s DIT will probably already saturate the detector if we use 16-bit ints. How exactly we deal with that is not clear to me.

According to the data rates document, the minimum DIT in the LM mode is 40 ms, and the minimum in the N band is 11 ms (but then NDIT=4). So it should be feasible to not saturate the detector simply by not doing such long observations.

[hugobuddel]

Done! See AstarVienna/ScopeSim#414 AstarVienna/irdb#170