Test code for flasher fractional error investigation

[SheridanLloyd]

I've refactored my test code to make it more parameterisable with different NSB rates, illuminations and calibration regimes and to factor out the common code to determine errors / moving averages. This code requires my get_flasher_illumination changes in source.py to run.

There are various methods to show that we get flasher output and that uniform illumination works as this was changed to accept flasher errors as we discussed before :
multiple_flasher_call, single_flasher_call_no_err, single_flasher_call_with_err, plot_multiple_flasher, plot_uniform_illumination

The most interesting methods are PlotFractionalError which subtracts average of 20 NSB runs from the measured and true charges as a pedestal and determines the fractional error (measured - true / true) from the moving average of measured and true . This recreates plots like those shown on the calls:

We can now also answer questions like "How many flashes reach a target fractional error ?" using GetNumberOfFlashesToReachTargetFractionalError. This is for a single pixel currently.

I can now specify a configurable observation regime that has the following steps (for a single camera pixel):

1)Specify number of NSB obs at a given rate to take and compute an average, "a pedestal" (i.ve tried 20 and 50 NSB obs)
2) Specify flasher illumination, pulse width and error. Take a moving average over an incrementing number n of flasher events with NSB as step 1 with pedestal subtracted. Verify that target fractional error is achieved for a configurable number of independent but consecutive "runs" each with n flashes.

NB Fractional error (FE) is defined as Measured charge - True Charge / True Charge. True charge is flasher charge without NSB . Measured charge is flasher+NSB but with pedestal subtracted

With this scheme I can get fractional errors of 1.5% (pushing this lower only works for low NSB ) at the 100 and 200 p.e. flasher illumination level for approx 100 flasher events across NSB up to 300 MHz and approx 150 events for 1000 MHz (moonlight) as my test outputs below:

"/home/sheridan/miniconda3/bin/python /home/sheridan/sstcam-simulation/test_flasher/call_flashers.py

Begin GetNumberOfFlashesToReachTargetFractionalError - no of NSB obs 20

Illumination,100,NSB,50,Target Fractional error,0.015 achieved for 10 runs in 33 flashes
Illumination,100,NSB,100,Target Fractional error,0.015 achieved for 10 runs in 24 flashes
Illumination,100,NSB,200,Target Fractional error,0.015 achieved for 10 runs in 19 flashes
Illumination,100,NSB,300,Target Fractional error,0.015 achieved for 10 runs in 90 flashes
Illumination,100,NSB,1000,Target Fractional error,0.015 achieved for 10 runs in 177 flashes
Illumination,200,NSB,50,Target Fractional error,0.015 achieved for 10 runs in 10 flashes
Illumination,200,NSB,100,Target Fractional error,0.015 achieved for 10 runs in 8 flashes
Illumination,200,NSB,200,Target Fractional error,0.015 achieved for 10 runs in 17 flashes
Illumination,200,NSB,300,Target Fractional error,0.015 achieved for 10 runs in 55 flashes
Illumination,200,NSB,1000,Target Fractional error,0.015 achieved for 10 runs in 156 flashes"

And for 2% FE target

Illumination,100,NSB,50,Target Fractional error,0.02 achieved for 10 runs in 18 flashes
Illumination,100,NSB,100,Target Fractional error,0.02 achieved for 10 runs in 18 flashes
Illumination,100,NSB,200,Target Fractional error,0.02 achieved for 10 runs in 25 flashes
Illumination,100,NSB,300,Target Fractional error,0.02 achieved for 10 runs in 29 flashes
Illumination,100,NSB,1000,Target Fractional error,0.02 achieved for 10 runs in 89 flashes
Illumination,200,NSB,50,Target Fractional error,0.02 achieved for 10 runs in 12 flashes
Illumination,200,NSB,100,Target Fractional error,0.02 achieved for 10 runs in 18 flashes
Illumination,200,NSB,200,Target Fractional error,0.02 achieved for 10 runs in 15 flashes
Illumination,200,NSB,300,Target Fractional error,0.02 achieved for 10 runs in 6 flashes
Illumination,200,NSB,1000,Target Fractional error,0.02 achieved for 10 runs in 73 flashes

There is variability in all this on successive runs - I am not implying these are the optimal number of flashes, just ballpark

We can discuss how this all makes it into a notebook - might need some guidance there

[SheridanLloyd]

Following good discussion with Jason on the next investigation, I've modeled the effect of decreasing over-voltage with increasing NSB, which Jon L. has shown decreases SiPM response to a laser pulse by 0.016% per MHz increase in NSB.

The method I've coded up is :

1. Initial pedestal subtraction at 50 MHz NSB
2. Increase NSB in a linear fashion 50 - 1000 MHz
3. Intersperce this NSB increase with flasher calibration runs where I measure the combined pedestal corrected flasher illumination + NSB ( "measured illumination" in attached plots), with each run having some number of flashes and flash rate
4. I simulate the decreased silicon response by dropping the flasher illumination, (so called "true illumination") depending on the NSB at the time of the flash
5. I determine the root mean square error (RMSE) to quantify the mis-calibration between "true" and "measured"

My code also now uses the common NSB pedestal subtraction and charge extraction routines in the simulator which should supresses NSB in relation to flash signal as it considers a narrow 20 ns window rather than whole waveform integration.

I attach some plots which show how "measured" and "true" vary with different flash rates, NSB rate of change, and number of flashes in each calibration run (which are ten minutes apart):
VaryFlasherRate_5_20_Hz_Initial_NSB_50MHz_100Flashes.log
VaryNo_Of_Flashes_Initial_NSB_50MHz_1.log
VaryNSB_ROC_Initial_NSB_50MHz_100_Flashes.log

The plots show a largely linear relationship between measured and true (as expected) whose linearity improves with number of flashes. The NSB rate of change plot is less useful as with increasing rate of change, we have fewer calibration runs and hence points on the plot, before getting to 1 GHz NSB.

I also generated some output log files (attached above) with my stats for these runs and the variable parameters for each run. In the files "MPRC" is "measured" and "TPRC" is "true". The take home message is that the mis-calibration RMSE varies from approximately 4 to 14 p.e. which for an initial flash of 100 p.e. can be taken as 4 to 14 % miscalibration which is within the stated requirement of 20 % . This is pretty much as expected from Jon L. figures (1000 MHz * 0.016 = 16 %). Increasing the number of flashes also takes the standard deviation on the measured calibration runs down from 7 p.e to <2 p.e. for 10 to 100 flashes in each calibration run respectively. Going to 200 flashes (files not attached) only takes standard dev down to around 1 p.e. so diminishing return.

I've also coded up a sin wave NSB generator so we can sample at any time within a max min NSB range with a given rise time and that is the next thing I will try to make sure the stats are not affected much by the level of the start NSB pedestal. This code will also improve on the linear NSB generator which stops at around 1 GHz and only allows me so much time to place the calibration runs in.

[SheridanLloyd]

PltVaryFlasherRateWithTrueIllumination,PltVaryNSBRateWithTrueIllumination,PltVaryNoOfFlashesWithTrueIllumination now can optionally take an NSB pedestal at the start of each inter-leaved calibration rather than once at the beginning of the entire run. The NSB increases linearly from 50 MHz to 1 GHz.

"Measured" (pedestal corrected with decreasing flasher illumination with NSB ) vs "True" ( The calculated decreased flasher illumination at the appropriate NSB) whilst varying flash rate, varying NSB rate of change and number of flashes in a calibration.

Standard deviation of measured :

RMSE is reduced from the maximum 14 % observed before with single pedestal to a 5-7% range when taking a pedestal at beginning of each interleaved flasher run: