No audio/TypeError: undefined is not an object (evaluating 'audio.currentTime = 0')

[jubejubess]

Hello!

Sorry for another audio-related question on my part. I'm programming an experiment wherein participants hear audio clips (via audio-keyboard-response) and respond via html-slider-response. The audio/response trials are set up in a timeline variable with a conditional function that inserts a break halfway through. I have two versions of the same experiment set up where the structure is exactly the same but just with different audio played. I seem to be encountering different audio issues with each experiment.

With one version, I always get 5-10 trials that play no audio at all and then goes straight to the html-slider-response trial. This seems to always occur during the latter half, never during the first half of the experiment. The audio files that don't play are at random, i.e. not always the same audio files are affected each time I run through. There's also no error messages in the console when this happens.

While for the other version (always after the first few trials), the audio suddenly doesn't play along with this error (which doesn't allow me to continue on with the rest of the experiment):

TypeError: undefined is not an object (evaluating 'audio.currentTime = 0')

I don't seem to encounter this issue with the other version, even though all of the audio clips come from the same source, just organized in different folders for each experiment/stimuli grouping. The audio was also playing fine when I had tested it a few weeks ago, but I'm not sure why it's not playing anymore (I've tested it on both my local JATOS install & the covid JATOS server).

I've tried to minimize any loading errors already by having all the audio clips preloaded in chunks throughout the experiment. I checked the network tab and I believe the audio clips are loading properly -- this is what shows up:

Any help or insight would be appreciated, thank you!

[jodeleeuw]

I'm stumped. Do you have a link to the experiment on the JATOS server that we could look at?

[jubejubess]

Hi, sorry for the delayed reply! Here's a link to the experiment on the JATOS test server: https://experiments.jatos.org/publix/1922/start?batchId=2273&generalMultiple

[becky-gilbert]

Hi @jubejubess - which experiment version does this link point to? The one with the TypeError: undefined error or the one without console errors?

[becky-gilbert]

I looked at the code and I'm really not sure what the problem is. Figured I'd share some thoughts, even though I'm kinda grasping at straws here...

For the TypeError: undefined is not an object error - is this the relevant bit of code where the error occurs?

var context = jsPsych.pluginAPI.audioContext();
    if(context !== null){
      var source = context.createBufferSource();
      source.buffer = jsPsych.pluginAPI.getAudioBuffer(trial.stimulus);
      source.connect(context.destination);
    } else {
      var audio = jsPsych.pluginAPI.getAudioBuffer(trial.stimulus);
      audio.currentTime = 0;           // <--- error here, right?
    }

If so, then I assume that audio is undefined because jsPsych.pluginAPI.getAudioBuffer function is sometimes returning undefined, which I assume happens because the audio file doesn't exist in the audio_buffers array:

module.getAudioBuffer = function(audioID) {

    if (audio_buffers[audioID] === 'tmp') {
      console.error('Audio file failed to load in the time allotted.')
      return;
    }

    return audio_buffers[audioID];

  }

@jodeleeuw from the code above, it looks like that if the audio file doesn't exist in jspsych.js's audio_buffers array then the audio plugin code will fail with this error (as opposed to trying to load the audio on-the-fly, which I think is what happens with image files). This would mean that the audio files have to be preloaded via jsPsych.pluginAPI.preloadAudioFiles, which is where the files are added to audio_buffers. Is that right?
If that's the case, then @jubejubess have you checked that there are no errors in your audio preload lists, and you're definitely preloading all of the audio files that are needed for each block?

And @jubejubess I see that you're preloading the audio in batches using jsPsych.pluginAPI.preloadAudioFiles in the call-function plugin, rather than via jsPsych.init.

var preloadaudio1 = {
  timeline_variables: audio1,
  data: {test_part: 'preload'},
  timeline: [
    {
      type: "call-function",
      async: true,
      func: function (done) {
        jsPsych.getDisplayElement().innerHTML = '<p>Loading experiment files... Please wait.</p>'
        jsPsych.pluginAPI.preloadAudioFiles([jsPsych.timelineVariable('audio', true)], function () {
          done({ preload: "success" });
        })
      }
    }
  ]
};

Could it be that the audio_buffers array is overwritten each time another batch is preloaded during the experiment, and maybe that can cause this problem if a preloading batch happens before one of the files from the previous batch is played?