List of issues with pupil benchmarking

[eozd]

Summary

This is a list of known issues with the pupil benchmarking, (examples/pspm_pupil) that needs to be addressed before the results are publishable

1. All the pupil files need to be reimported using the new Eyelink import with no filtering during import

• In addition to pupil, gaze and marker channels, blink and saccade channels also need to imported so that blink/saccade filtering can be applied
• Once this is done, files should be put to examples/pspm_pupil/data in the current structuring.

2. Sometimes pspm_glm crashes due to the inability of pspm_prepdata to handle NaN values. The crash logs can be found in experiment 2 logs in examples/pspm_pupil/log/exp2 folder in science cloud
3. Valid fixations filtering in experiment 7 seems to be crashing. This issue needs to be handled, similarly by checking the logs in examples/pspm_pupil/log/exp7.
4. Some Cohen's D scores seem to be negative. Although this is theoretically possible, I am not sure if this should be happening at all. This issue needs to be investigated
5. In experiment 1, given a dataset, score values seem to be constant with respect to different parameter combinations. (Should be fixed with PsPM PR @ Mult NaN ratio with 100 in extract_segments bachlab/PsPM#108)
6. Some score values are returned as NULL; however, this is not a bug. When the exclusion threshold is too low, none of the subjects in a dataset are fitted.
7. Recording geometry should be set properly in examples/pspm_pupil/libcommon/pp_pfe; currently it is set to one of the auto modes for convenience.