Backward masking of scrambled images

[VL914]

Hello,

I'm trying to construct an experiment that uses scrambled images for backward masking. Is there any way to scramble images dynamically in each trial, instead of using pre-scrambled images, with jsPsych? Because we are trying to match the scrambled maskers with the stimuli one-to-one in each trial, and it would be hard to pre-scramble every image beforehand since we have a large dataset of images. It would be greatly appreciated if you can offer some suggestions. Thank you!

[jodeleeuw]

Hi @VL914 ,

What kind of scrambling approach do you use? Can you post an example of a scrambled an unscrambled image?

[jodeleeuw]

If you have access to or know how to write an algorithm that will take image data and perform this manipulation on it, then you could use the <canvas> element to display the image and run a custom filter on it. Here's a tutorial in non-jspsych context: https://www.html5rocks.com/en/tutorials/canvas/imagefilters/.

The image-*-response plugins in jsPsych 6.2 will render images on a canvas by default, so it should be possible to do this filtering within those plugins.

[VL914]

Hi @jodeleeuw, thanks much for your reply! I tried several code snippets to work with the image filter method as you suggested above, but none of them worked. I'm actually turning to a simpler scrambling method, which is just randomly scrambling the pixels of each image, rather than the phase scramble.
I'm trying to use this p5.js library to get the pixel data of images. But I don't know how to apply this library with jsPsych.

Here's my code for stimulus and masking screens. Because p5.js draws an image on the canvas, so I use a canvas- plugin instead of the image- ones.

//stimulus
        {
            type: 'image-keyboard-response',
            stimulus: jsPsych.timelineVariable('pic'),
            maintain_aspect_ration: true,
            choices: jsPsych.NO_KEYS,
            trial_duration: function(){
                soaNow = stim_duration_dic[trialCount]
                return soaNow
            },
            prompt: function () {
                return "<br><br><h5>Presentation time: " + soaNow + "</h5>"
            },
            data: { pic: jsPsych.timelineVariable('pic') },
        },
//masking
{
            type: 'canvas-keyboard-response',
            stimulus: imgScramble ,
            choices: jsPsych.NO_KEYS,
            trial_duration: 1000
},
//response
....

This is the imgScramble function. I'm using the "instance mode" as suggested here and here. The code inside setup() is not for scrambling, but a sample snippet I took from here

 function imgScramble (c){
        c.getContext('bitmaprenderer')
        lastimg = jsPsych.data.get().last(1).values()[0].pic
        
        c.setup = function(){
            myImage = loadImage(lastimg);

            myImage.loadPixels()
            halfImage = 4 * myImage.width * myImage.height / 2;
            for (let i = 0; i < halfImage; i++) {
                myImage.pixels[i + halfImage] = myImage.pixels[i];
            }
            myImage.updatePixels();
        }
        
        c.draw = function(){
            image(myImage, 0, 0, width, height);
        }
    }

The result is that there's no error popping up in the console, but no image is displayed in the stimulus screen (only prompt is shown), and the masking trial seems to be directly skipped (not enough 1000ms gap between stimulus and response trial). I'm not sure if I'm overwriting the stimulus screen with some blank canvas. It would be greatly appreciated if you can provide some suggestions on how to use p5.js within jsPsych, or any other method that can manipulate image pixels in jsPsych. Thank you very much!