Scoping CHS jsPsych studies

[becky-gilbert]

This issue is to track the multiple smaller issues that will need to be implemented in order to introduce support for jsPsych studies on CHS, and to manage the prioritization of optional/'nice to have' features over time and/or release phases.

jsPsych phase 1 MVP

Context

Several of our major grants have 'supporting mobile studies' as a deliverable or key prerequisite; we provide a limited version of this in the form of external studies, but fleshing out to full support is a primary goal on our roadmap. In looking at the best way to accomplish this, we determined that importing support for jsPsych is preferable to building out a new major feature set in the EFP. This will allow us to take advantage of existing code, and existing documentation and open source community that CHS researchers will be able to use.

As with any user-facing feature we build, documentation, templating, and active community support resources are first-class citizens; because jsPsych already provides a lot of this we will try to focus on things that are specific for CHS use and aren't duplicated in existing jsPsych resources.

Things to keep in mind:

• The most important goal of adding jsPsych is to support mobile studies. This should be kept in mind for feature prioritization, testing, and documentation.
  • Other goals/major upsides include long-term migration to a more stable, community-maintained open source experiment builder for all studies, to improve the sustainability of the CHS project...but mobile support is primary!!
• The aim of the initial MVP is to release a 'beta' feature that allows:
  • Families to be recruited to, participate in, and afterward view their session from at least one kind of study paradigm without leaving their mobile device, and without seeing any feature regressions compared to a (theoretical) EFP internal study.
  • Researchers to fully implement a jsPsych study hosted on CHS, and use the main tools of CHS for experiment administration the way other internal users do.
  • It is okay if this requires researchers to already be very comfortable writing jsPsych, or to take extra steps/adapt their workflows when working with data produced with jsPsych vs. ember-frame-player.
• Future MVPs should target making the jsPsych experience at least as accessible as the Lookit EFP, including documentation and study templates, and reductions in anything 'weird' researchers need to do to handle jsPsych experiments.
  • Note that making a general change to CHS that affects researchers in terms of how both EFP and jsPsych work is fair game and will likely be required!
• For sustainability, whenever the choice is available to do things in the 'vanilla jsPsych way' and teach/document for CHS users how to use it, we should make that choice.

User stories

Keep these in mind for testing and scoping of individual areas of work!

• A researcher creates the study using the Study Creation form, edits on Edit, previews and views resulting data, submits for admin approval. Admin previews study, study is approved and launches. Researcher reviews & assigns consent, reviews individual study responses, downloads datasets, communicates with participants.
• A family starting up the study, completing (or not completing), researcher assigns consent, reviews study content, pays participant, family views their previous/completed study.
• From a family perspective, the JSPsych/Lookit engine should be invisible from a functional perspective. This does NOT meant that JSPsych studies need identical look & feel to particular Lookit frames; it does mean that families should not need to learn/do anything different to be successful during their first study using the Lookit vs. jsPsych engines.

User requirements - Researchers

Phase 1 frame list

Make these available roughly as with EFP, i.e. a jsPsych or CHS-jsPsych plugin with modifiable content is available to researchers, layout & etc. controlled by us, and with impacts on lookit-api

• Video consent frame (exp-lookit-video-consent) - With most up-to-date and parametrized single template: Create a CHS-jsPsych video-consent plugin #1295
• Survey consent frame (exp-lookit-survey-consent): Create a CHS-jsPsych survey-consent plugin/template #1328
• Exit survey (exp-lookit-exit-survey) (with new Databrary language??): Create a CHS-jsPsych exit survey plugin #1296
• Webcam setup (exp-lookit-video-config): Create a CHS-jsPsych video-config plugin #1297 <- note this is on this list not because of api impact, but because we've worked hard to make this work for families; keep what we've learned rather than having each lab need to write it themselves.
• Video recording extension (EFP video recording mixin and trial/session recording functionality): JsPsych Record Video lookit-jspsych#35

Other frames (optional)

• Video assent frame (exp-lookit-video-assent) - The same, for use with children over ~age 8: Create a CHS-jsPsych video-assent plugin #1329

For any other frames we try to provide to the users in the initial MVP, consider carefully (for both workload and outcome) whether to:

• Point users to an existing jsPsych plugin and document the differences to the nearest equivalent
• Provide template code with best practices showing how to reproduce the behavior out of simple templates - e.g. rather than duplicating the feature-heavy "exp-lookit-images-audio" as a custom CHS-jsPsych plugin, provide training/templates showing how to construct the parts you want using jsPsych's existing plugins (e.g. how to display a stimulus, add a parent information bar below, add/remove timing for moving on, etc.)
• Create a custom plugin with CHS-specific parameters

Candidate frames for this (do no more than 2-4 total) include:

• exp-lookit-calibration (jsPsych has the webgazer calibration plugin, but this is meant for use in conjunction with webgazer eyetracking and is dependent on loading webgazer. If the jsPsych calibration plugin is written in a modular way, we might be able to extract the relevant stimulus presentation code/parameters).
• exp-lookit-text (i.e. for an instructions page) <- good example of a frame where we'd want to show the vanilla jsPsych option (with jsPsych's HTML keyboard/button plugins), plus how to get the desired "blocks and title" structure with html code, showing the parallel code to produce the same/very similar result.
• exp-lookit-images-audio - as above: show how to convert from EFP's images-audio to jsPsych's image and audio plugins
• exp-lookit-survey - as above: demonstrate qualtrics-like functionality and show how to convert from EFP's survey frame to jsPsych's survey and survey-* plugin options

Feature implementation:

This is the list of minimum requirements for the initial 'phase 1' release of jsPsych study support:

• Add jsPsych to study types: jsPsych Study Type #1262
• Add jsPsych views and URLs for study participation/preview: jsPsych Study Type #1262
• Decide on jsPsych URL format: Study participate/preview URLs for different experiment runners #1325
• Include a single core jsPsych version and minimal set of automatically-available official jsPsych plugins: jsPsych Study Type #1262 Note: need to update to latest releases
  • jsPsych core v7.3.3
  • html-keyboard-response v1.1.2
  • image-keyboard-response v1.1.2
  • preload v1.1.2
• Automatically include a CHS-jsPsych video config plugin: Create a CHS-jsPsych video-config plugin #1297 video-config plugin lookit-jspsych#45
• Automatically include a CHS-jsPsych video consent plugin: Create a CHS-jsPsych video-consent plugin #1295 Video consent trial lookit-jspsych#52
• Automatically include a CHS-jsPsych survey consent plugin: Create a CHS-jsPsych survey-consent plugin/template #1328 Jspsych Surveys lookit-jspsych#24
• Automatically include a CHS-jsPsych exit survey plugin: Create a CHS-jsPsych exit survey plugin #1296 Jspsych Surveys lookit-jspsych#24
• Automatically include CHS-jsPsych video recording plugins/extension so that videos can be recorded and stored in our S3 bucket: JsPsych Record Video lookit-jspsych#35, Lookit's jsPsych Video Extensions and Plugins lookit-jspsych#25
• Create a lookit/CHS-jsPsych monorepo to store our custom versions of initJsPsych and custom plugins/extensions: Create CHS-jsPsych monorepo #1327
• Pass the child/session info into jsPsych studies (as in protocol generator): jsPsych studies should receive child/session info #1294
• Send jsPsych data to lookit-api via response API and mark completed responses (using custom Lookit version of initJsPsych): CHS jsPsych init function #1282, jsPsych Response Data #1283, jsPsych Study Type #1262
• Redirect jsPsych studies to the exit URL after saving data: jsPsych Study Type #1262
• Support language translations for CHS-jsPsych plugins: Support translations lookit-jspsych#53

Lower priority / if there's time:

• Check the jsPsych trials array for CHS required trials (using custom Lookit version of initJsPsych)
• Automatically include a CHS-jsPsych video assent plugin: Create a CHS-jsPsych video-assent plugin #1329
• Add the ability to pause studies to CHS-jsPsych studies: Add default pausing functionality to CHS-jsPsych studies #1299
• Jest tests

Changes to lookit-api views:

• Integrate jsPsych data with consent manager: Update consent manager view for jsPsych responses #1332 jsPsych Consent Manager #1457
• Integrate jsPsych data with response views: All Responses (download options) and Individual Responses (downloads and info displayed to resarchers): Integrate jspsych responses with app UI #1316 Jspsych response helpers #1454
• Integrate separate jsPsych S3 bucket for viewing/downloading videos (researcher downloads, family study history page): Support separate S3 bucket for jsPsych videos #1442 jsPsych video/data access #1465

Documentation:

• Add a jsPsych page/section in the CHS Lookit docs (with lots of links to the jsPsych docs!)
• Add/modify docs to go along with any of the more specific changes and new features listed above
• Update the Lookit architecture documentation page: Update Lookit architecture docs after adding support for other exp runners lookit-docs#319
• One basic example showing how to convert a basic CHS Lookit study protocol into a jsPsych study (i.e. mapping terminology such as frames/plugins, kind/type, etc.)
• One basic example showing how to randomize/counterbalance with jsPsych
• Specify the automatically available jsPsych core package version, list of plugins (jsPsych and CHS-jsPsych) and their versions. All versions will be fixed to start with - we will add the ability to change versions in a later release.

In and Out

There is a big range in the priority/urgency across these issues, so we are making some rulings about what is going to be tackled in the first release MVP for jsPsych! (Melissa has sorted these into bins based on standup meeting 12/12.)

In for this MVP

• CHS-jsPsych video config plugin: Create a CHS-jsPsych video-config plugin #1297
• CHS-jsPsych video consent plugin: Create a CHS-jsPsych video-consent plugin #1295
• CHS-jsPsych survey consent plugin: Create a CHS-jsPsych survey-consent plugin/template #1328
• CHS-jsPsych exit survey plugin: Create a CHS-jsPsych exit survey plugin #1296
• Start/stop video recording and video uploading: JsPsych Record Video lookit-jspsych#35
• Pass the child/session info into jsPsych studies (as in protocol generator): jsPsych studies should receive child/session info #1294
• Automatically provide a single jsPsych core version and a limited set of plugins with fixed versions - researchers will not have the ability to change core/plugin versions (and we will not upgrade versions on our end either).
• Basic CHS jsPsych docs

Lower priority / if there's time:

• CHS-jsPsych video assent plugin: Create a CHS-jsPsych video-assent plugin #1329
• Add the ability to pause studies to CHS-jsPsych studies: Add default pausing functionality to CHS-jsPsych studies #1299

Out for this MVP

The following is a list of things we are consciously NOT including in the jsPsych phase 1 MVP, and are punting to some future time.

Site/UI features:

• Providing researchers with access to more/all official jsPsych plugins/extensions (either via the UI or automatically loading them)
• Allowing researchers to (up)load their own custom plugins/extensions
• Allowing researchers to select from other versions of the core jsPsych library and plugins. (This will require listing the available packages and versions for the user to select via the study create/edit UI, recording the version information with the study info in the database, and loading package versions dynamically)
• Dependency management tools, e.g. validating the compatibility between the core jsPsych package and plugin versions

Plugins/extensions:

• CHS-jsPsych video config quality plugin
• CHS-jsPsych extension for infant-controlled timing
• CHS-jsPsych stimuli preview plugin
• CHS-jsPsych observation plugin
• CHS-jsPsych change detection plugin
• CHS-jsPsych calibration plugin

Documentation:

• More jsPsych code FAQ/examples
• Study template examples
• More extensive support on randomization/counterbalancing. Probably this should be a single study, implemented two ways:
  • Use the beloved-of-developmental-psychologists list approach: "For each participant, choose and run one of these lists: [condA_stim1, condA_stim2, condA_stim3]; [condA_stim3, condA_stim1, condA_stim2]; [condA_stim2, condA_stim3, condA_stim1]; condB_stim1, condB_stim2, condB_stim3]; [condB_stim3, condB_stim1, condB_stim2]; [condB_stim2, condB_stim3, condB_stim1]"
  • Use fully random assignment: "For each participant, flip a coin for condition A or B. For each participant, randomize the order of 3 stimuli, and show the condition-appropriate versions of each stimulus in the appropriate order."