feature request: downsampling

[shawntz]

• will need to do the closest match between timestamps to snap the event messages to after downsampling
• add to glassbox as option: when downsampling in the glassbox function, place it as optional right after lpfilt() but right before the optional detrend()

[hyang336]

What was this function intended for again?

[shawntz]

@hyang336 this function is intended to be a downsampling method -- i.e., bin timeseries into x-ms intervals (in case the user doesn't want the raw data out at 1000Hz (or whatever the tracker recorded at)

[shawntz]

@mh105 @hyang336 @alicexue - quick follow-up question on this: should the binning (ie downsampling) step be its own step, or baked into the epoching step as a flag (or both)? i'm wondering whether folks would be interested in downsampling the data prior to epoching, or if they'd just want to have a more coarse binning of the epoched data.

[mh105]

If by binning you are referring to averaging during chunks of trial periods, then it's not really downsampling. If you are indeed referring to downsampling (i.e., like how 1000Hz eye tracking data can be downsampled to 100Hz), then it should be renamed as downsample instead of bin to remind people that they can't just decimate their data without worrying about aliasing.

Which one of these two routes are you thinking? We may potentially want both implemented.

[shawntz]

this is a super helpful distinction, thanks — i think we probably want both. it would be helpful to have some more context about when you’d want to do either, since i don’t typically use either of these when analyzing tonic data (but want to provide flexibility for folks who primarily do ERP-like analyses) sent from my iphone

…

On Tue, Sep 3, 2024 at 1:06 PM Mingjian He ***@***.***> wrote: If by binning you are referring to averaging during chunks of trial periods, then it's not really downsampling. If you are indeed referring to downsampling (i.e., like how 1000Hz eye tracking data can be downsampled to 100Hz), then it should be renamed as downsample instead of bin to remind people that they can't just decimate their data without worrying about aliasing. Which one of these two routes are you thinking? We may potentially want both implemented. — Reply to this email directly, view it on GitHub <#18 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABHVTCTRAJCWWUWXNZGJDCLZUYJDHAVCNFSM6AAAAABNA66MSGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRXGMZTSOJYGM> . You are receiving this because you were assigned.Message ID: ***@***.***>

[mh105]

Personally I think there are better ways to achieve both binning and downsampling. But I agree that binning gets used quite a bit in the literature when people want to study the "temporal dynamics" of pupil dilatory response. Bona fide downsampling is rarer, because few people do temporal modeling of pupil data where averaging within bins will distort the dynamical system.

My suggestion is to provide both as separate pipeline functions that people can use, but you should be very clear about their distinctions in function header / documentations.

• binning is dividing one second of pupillary data into X bins and averaging pupillometry data around each bin center. Therefore after binning, the time points with pupillary data will be: 1/2X, 3/2X, 5/2X, ..., etc.
• downsampling is decimating one second of pupillary data into X samples. No averaging will be done, and anti-aliasing filter is applied before downsampling. So the time points with pupillary data after downsampling will be: 0, 1/X, 2/X, 3/X, ..., etc.

binning is straightforward, just need to be careful about using the bin center as the next time point. downsampling on the other hand needs to be very very careful about designing the anti-aliasing filter. You can use the lpfilt() function to do so, but this downsampling function should specify the wp, ws, rp, rs. Here's what I would do.

Say the new desired sampling frequency after downsampling is Fs_new. Then the Nyquist is Fs_nq = Fs_new / 2. I would set ws = Fs_nq, then decide on wp as the following pseudo-code:

wt <- max(5, Fs_nq * 0.2)
wp <- ws - wt
# raise an error if wp < 4 because that means actual pupillary responses are likely lost during this downsampling

Then set rp = 1 and rs = 35 as in lpfilt().