From 3c162112f7725cc628b69c98b58d916632056cf9 Mon Sep 17 00:00:00 2001 From: Becky Gilbert Date: Mon, 4 Nov 2024 13:53:25 -0800 Subject: [PATCH 01/24] first pass at adding jsPsych experiment runner, clarifying CHS vs Lookit, switching to CHS email and URLs --- README.md | 14 +++-- docs/source/community-involvement.rst | 44 ++++++++-------- .../community-participant-recruitment.rst | 14 ++--- .../community-study-approval-process.rst | 40 +++++++------- ...munity-study-review-basic-requirements.rst | 6 +-- ...community-study-review-checklists-self.rst | 22 ++++---- docs/source/contribute-edit-docs.rst | 2 +- docs/source/contribute-github-issue.rst | 14 ++--- docs/source/contributor-guidelines.rst | 26 ++++++---- docs/source/faq.rst | 50 ++++++++++-------- docs/source/features.rst | 25 +++++---- docs/source/frame-dev-creation.rst | 4 +- docs/source/frame-dev-setup.rst | 20 +++---- docs/source/install-ember-app.rst | 10 ++-- docs/source/other-learning-materials.rst | 22 +++++--- docs/source/publications.rst | 10 ++-- docs/source/researchers-create-experiment.rst | 14 ++--- ...researchers-day-to-day-study-operation.rst | 26 +++++----- docs/source/researchers-experiment-data.rst | 10 ++-- docs/source/researchers-lag-issues.rst | 12 ++--- docs/source/researchers-log-in.rst | 52 +++++++++---------- docs/source/researchers-manage-consent.rst | 14 ++--- docs/source/researchers-manage-org.rst | 22 ++++---- docs/source/researchers-prep-stimuli.rst | 2 +- docs/source/researchers-set-study-fields.rst | 30 +++++------ docs/source/researchers-spam-prevention.rst | 16 +++--- docs/source/researchers-study-permissions.rst | 6 +-- docs/source/researchers-style-guide.rst | 4 +- docs/source/researchers-update-code.rst | 4 +- docs/source/researchers-use-api.rst | 34 ++++++------ docs/source/tutorial-access.rst | 40 +++++++------- docs/source/tutorial-contributing.rst | 22 ++++---- docs/source/tutorial-first-study.rst | 30 +++++------ docs/source/tutorial-manage-data.rst | 20 +++---- docs/source/tutorial-next-steps.rst | 15 +++--- docs/source/tutorial-protocol-exercises.rst | 8 +-- docs/source/tutorial-second-study.rst | 18 +++---- docs/source/vision-for-lookit.rst | 4 +- snippets.rst | 4 +- 39 files changed, 378 insertions(+), 352 deletions(-) diff --git a/README.md b/README.md index 507a5016..31e1eab7 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,14 @@ # lookit-docs -Documentation for Lookit project, including use of platform and lookit-api and ember-lookit-frameplayer repos. +Documentation for Children Helping Science (formerly "Lookit") project, including use of the CHS platform, the internal Lookit and jsPsych experiment runners, and associated repositories. -Lookit platform: https://lookit.readthedocs.io/en/develop/ +CHS platform: https://lookit.readthedocs.io/en/develop/ -Ember Lookit Frameplayer (experiment runner): https://lookit.readthedocs.io/projects/frameplayer/en/latest/ +Experiment runners: + +Lookit (Ember Lookit Frameplayer): https://lookit.readthedocs.io/projects/frameplayer/en/latest/ + +jsPsych: https://lookit.readthedocs.io/projects/chs-jspsych/en/latest/ --- @@ -12,7 +16,7 @@ Ember Lookit Frameplayer (experiment runner): https://lookit.readthedocs.io/proj Have you spotted an error in the documentation, or have other suggestions for improvement? We'd love your contributions! If you're not sure whether we'll want to incorporate your changes, please open an Issue in this repository so that we can discuss it first. -If you are not part of the Lookit team and would like to contribute changes to the Lookit documentation, you will follow the same Installation and Editing steps listed below except that you will *make changes on your own fork of this repository*. Here is a summary of the process: +If you are not part of the CHS/Lookit team and would like to contribute changes to the documentation, you will follow the same Installation and Editing steps listed below except that you will *make changes on your own fork of this repository*. Here is a summary of the process: 1. Fork this repository. 2. Set up your local environment, following the Installation steps below. 3. Make your changes, commit them, and push to your fork/branch. @@ -298,7 +302,7 @@ Below are references for embedded/reusable links and footnotes (see https://gith [sphinx-rst]: https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html [^forkfootnote]: - If you are not part of the core Lookit team then you should *fork this repository* first (fork button in upper-right corner of this page), then clone your fork: + If you are not part of the core CHS/Lookit team then you should *fork this repository* first (fork button in upper-right corner of this page), then clone your fork: ``` / $ git clone https://github.com//lookit-docs.git / $ cd lookit-docs diff --git a/docs/source/community-involvement.rst b/docs/source/community-involvement.rst index 85604aad..8ed6e8b5 100644 --- a/docs/source/community-involvement.rst +++ b/docs/source/community-involvement.rst @@ -1,11 +1,11 @@ Community involvement ================================================ -Currently Lookit is run by three people working at home during a +Currently the Children Helping Science (CHS) project is run by three people working at home during a pandemic. We had a description here of all the things we do, but really, enough said. -Ways *all* Lookit researchers contribute +Ways *all* CHS researchers contribute ---------------------------------------- 1. Peer-to-peer support @@ -18,27 +18,27 @@ that were confusing! 2. Provide peer feedback on studies ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Before submitting a study to Lookit, researchers gather peer feedback +Before submitting a study to CHS, researchers gather peer feedback (see :ref:`Study approval process `). This is a quick and lightweight process, not scientific peer review, but it does a lot to improve the participant experience! As a researcher on -Lookit, you will help to review other studies as well as posting your +CHS, you will help to review other studies as well as posting your own for feedback. 3. Recruitment ~~~~~~~~~~~~~~ Once you start running your study, you will need to actively recruit -participants. There’s a shared Lookit participant pool, but we need +participants. There’s a shared CHS participant pool, but we need everyone’s combined efforts to build it. If you have programming experience (or employ someone who does) --------------------------------------------------------------- -Lookit is an open-source project, but struggles with the problem that relatively few of +CHS is an open-source project, but struggles with the problem that relatively few of its primary stakeholders (developmental researchers) feel equipped to contribute to feature development. If you have experience programming in Python, the most helpful way -to contribute is to :ref:`Set up for local development ` on the Lookit codebase, take +to contribute is to :ref:`Set up for local development ` on the CHS codebase, take ownership of an issue in the lookit-api repo and make a PR with your implementation (and your unit tests ;) ). @@ -65,9 +65,9 @@ The existence of the Working Groups has many advantages, including: and only by those who (e.g.,) happen to be paying attention to a general email listserv at that time. -Anyone involved with a current study on Lookit, or who is developing +Anyone involved with a current study on CHS, or who is developing one, is very welcome to join a Working Group! We also have an -expectation that **any research team hosting a study on Lookit will have +expectation that **any research team hosting a study on CHS will have at least one member who is a member of at least one Working Group**, with a flexible time commitment of about 2 hours/week. The person meeting this requirement for their research group should be a “main” @@ -80,7 +80,7 @@ Here are the current working groups: Best Practices for Video Chat Studies -------------------------------------------------------------------------------- -The goal of this Working Group is to develop best practices (and, rarely, rules) about running video chat studies (also called "synchronous" studies, these are typically "Zoom" studies). Over the past couple of years, there have been many groups independently identifying great ways to do video chat studies, and we hope to discuss and iterate on what a diverse group of researchers have done. As we develop resources for best practices, members of Lookit (and others) may choose to follow them because they are ideas that have been refined over time by a large and diverse group of people, and also just because consistency can be inherently valuable (e.g., even if a practice is not inherently better than some alternative, being consistent with our participants can make things easier and more predictable for them). See a draft of some ideas `here `_. +The goal of this Working Group is to develop best practices (and, rarely, rules) about running video chat studies (also called "synchronous" studies, these are typically "Zoom" studies). Over the past couple of years, there have been many groups independently identifying great ways to do video chat studies, and we hope to discuss and iterate on what a diverse group of researchers have done. As we develop resources for best practices, members of CHS (and others) may choose to follow them because they are ideas that have been refined over time by a large and diverse group of people, and also just because consistency can be inherently valuable (e.g., even if a practice is not inherently better than some alternative, being consistent with our participants can make things easier and more predictable for them). See a draft of some ideas `here `_. Recruitment (social media, etc.) -------------------------------------------------------------------------------- @@ -98,17 +98,17 @@ Eliciting and organizing feature feedback from researchers The goal of this Working Group is to gather information from researchers in multiple ways (e.g., surveys, focus groups, monitoring of questions -on Slack and GitHub issues) about potential new features for Lookit, and +on Slack and GitHub issues) about potential new features for CHS, and work towards prioritizing them. This requires synthesizing information about what would best benefit researchers and what is easiest for the programmers to implement, and so this group will work with the programmers as well. -Advise and support new Lookit researchers (“Researcher onboarding”) +Advise and support new CHS researchers (“Researcher onboarding”) -------------------------------------------------------------------------------- The goal of this Working Group is to help new researchers who are -designing their first Lookit study. This can involve creating additional +designing their first CHS study. This can involve creating additional tutorials/samples/documentation, providing 1-on-1 mentoring, and monitoring a Slack channel where new researchers ask questions. @@ -116,7 +116,7 @@ Parent-facing “first 10 minutes” -------------------------------------------------------------------------------- The goal of this Working Group is to maximize the parent-facing -experience, from finding Lookit to finishing the consent for the child’s +experience, from finding CHS to finishing the consent for the child’s first study. Example work might include revising the homepage and doing focus group (user experience) studies of naive parents navigating it, creating informational videos about what participation entails and @@ -138,15 +138,15 @@ Researcher-facing “first 10 minutes” -------------------------------------------------------------------------------- The goal of this Working Group is to maximize the researcher-facing -experience, from discovering Lookit to deciding whether or not it might +experience, from discovering CHS to deciding whether or not it might be a good fit for their research. For example, this group might create a webpage that is directed towards PIs, to determine in 3-5 minutes -whether joining Lookit might be a good use of a graduate student’s time. +whether joining CHS might be a good use of a graduate student’s time. Another webpage might be directed towards graduate students, to get a -sense of the types of research Lookit is a good fit for, and what types -of skills would be necessary to conduct that research via Lookit. Much +sense of the types of research CHS is a good fit for, and what types +of skills would be necessary to conduct that research via CHS. Much of this information already exists, but optimizing it for different -audiences could make a big difference in the uptake of Lookit by +audiences could make a big difference in the uptake of CHS by research teams. The A-Team @@ -154,11 +154,11 @@ The A-Team Named after a TV show about mercenaries for hire: “If you have a problem, if no one else can help, and if you can find them, maybe you -can hire… the A-Team.” For Lookit, this Working Group will work on all +can hire… the A-Team.” For CHS, this Working Group will work on all of the miscellaneous tasks that are shorter term (so they don’t have their own working group) but totally essential for the development of -Lookit. Maybe a couple of weeks working on legal and ethical issues to -build a knowledge base for getting Lookit access agreement signed. And +CHS. Maybe a couple of weeks working on legal and ethical issues to +build a knowledge base for getting CHS access agreement signed. And then the next project might be exploring non-monetary compensation for families (certificates? personalized information?). All skill sets welcome for a well-balanced group of problem solvers. diff --git a/docs/source/community-participant-recruitment.rst b/docs/source/community-participant-recruitment.rst index 1884f7b2..8ff152f7 100644 --- a/docs/source/community-participant-recruitment.rst +++ b/docs/source/community-participant-recruitment.rst @@ -10,7 +10,7 @@ See `this Google doc `__). Put up flyers and/or talk to local institutions/people that work with - kids to get them excited about Lookit, ask them to mention it to + kids to get them excited about CHS, ask them to mention it to families, maintain relationships. Examples of places we've looked when advertising locally in Cambridge: @@ -71,7 +71,7 @@ Potential approaches: - Pediatricians’ offices - Parks - Daycares, preschools, elementary schools, afterschool programs - - MIT museum (not because they get a ton of kids but because we already have a connection via MIT, and in the future maybe they’d like to host an interactive Lookit display...) + - MIT museum (not because they get a ton of kids but because we already have a connection via MIT, and in the future maybe they’d like to host an interactive CHS display...) - `Cambridge Family Resource Center `__ - `Baby U, Center for Families playgroups, other parent ed `__ @@ -88,9 +88,9 @@ Potential approaches: - Media/online outreach - - Talk to groups that do online research with adults (e.g. LabInTheWild, TestMyBrain) to see if they’d be up for linking to Lookit + - Talk to groups that do online research with adults (e.g. LabInTheWild, TestMyBrain) to see if they’d be up for linking to CHS - Talk to groups that do online research with kids - e.g. https://www.babysleepstudy.org/studysignup, - - Ask institutions that might be interested about featuring/linking to Lookit, or publishing content we provide - examples: + - Ask institutions that might be interested about featuring/linking to CHS, or publishing content we provide - examples: - Boston Children’s Museum - https://www.parentingscience.com/online-parenting-studies.html diff --git a/docs/source/community-study-approval-process.rst b/docs/source/community-study-approval-process.rst index ef19b412..c38425a0 100644 --- a/docs/source/community-study-approval-process.rst +++ b/docs/source/community-study-approval-process.rst @@ -3,20 +3,20 @@ Study approval process ~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Remember, before your study can go live on Lookit, there are three things you'll need to do: +Remember, before your study can go live on CHS, there are three things you'll need to do: 1) :ref:`Sign and return the access agreement.` This covers your entire lab. You only have to do this once. 2) :ref:`Get IRB approval for your study at your own institution.` -3) Get peer feedback on the study, and then submit it to Lookit for approval. This is the part we'll discuss here! +3) Get peer feedback on the study, and then submit it to CHS for approval. This is the part we'll discuss here! Why is there a study review process? ============================================== -Because Lookit is a shared platform with a shared reputation, it -benefits all of us to uphold high standards for all Lookit studies. +Because CHS is a shared platform with a shared reputation, it +benefits all of us to uphold high standards for all CHS studies. Families who have a good experience - the study is fun, it runs smoothly, the instructions are easy to follow, they get a gift card promptly or get quick friendly answers to their questions, their child @@ -25,7 +25,7 @@ likewise benefits all of us. To make sure all studies meet community standards, we use two forms of review before studies are posted: another researcher -outside your lab providing feedback ("peer review"), and Lookit staff reviewing +outside your lab providing feedback ("peer review"), and CHS staff reviewing studies before approving them for posting ("admin review"). While this may feel like "extra steps" compared with in-lab research, it's an opportunity to improve both the participant @@ -37,9 +37,9 @@ see how much their studies improve in the process. What are the rules for external studies and 'live' sessions? ============================================================= -From the beginning of Lookit through most of 2021, all studies listed on Lookit -were also run within Lookit, using the Lookit experiment runner. In contrast, -the new "External Study" type recruits participants via Lookit but the +From the beginning of CHS through most of 2021, all studies listed on CHS +were also run using the Lookit experiment runner. In contrast, +the "External Study" type recruits participants via the CHS site but the experiment itself takes place elsewhere. This opens up the possibility for researchers to run "asynchronous" studies with any desired platform (e.g., Qualtrics) or "synchronous" studies (e.g., scheduled Zoom sessions). @@ -51,7 +51,7 @@ admin review will be expedited; however, all external studies must still meet :ref:`basic requirements`. You are also strongly encouraged to review the -:ref:`study guidelines` that we use for reviewing Lookit experiments, +:ref:`study guidelines` that we use for reviewing internal Lookit and jsPsych experiments, and are always welcome to give and solicit peer review for any study type on Slack. This is probably the best way to learn about what features make studies successful on our platform, and we recommend that all new researchers contribute peer reviews so that @@ -64,7 +64,7 @@ If you don't have IRB approval or an access agreement yet ========================================================= You can still go ahead with steps 1 and 2 below so you're ready to go as -soon as possible! The "Lab" your study is associated with on Lookit will +soon as possible! The "Lab" your study is associated with on CHS will have to be approved to test before you'll be able to submit in Step 3, which means you'll need the access agreement set at that point. @@ -80,7 +80,7 @@ before requesting peer review! Step 2: peer review ===================== -Next, post a preview link for your study on the Lookit Slack #researchers channel to +Next, post a preview link for your study on the CHS/Lookit Slack #researchers channel to gather some feedback from researchers *outside your group*. This is like getting a bunch of fresh eyes at a lab meeting made up of researchers who work online! Check out the Slack channel to see many previous examples of how to post review requests, @@ -90,11 +90,11 @@ other recently posted studies - trading reviews in this way is how we keep the s The goal of peer review is to have someone go through the study *exactly as a family browsing CHS would do*. This means that you should always direct reviewers to -the Lookit preview link, even if your study takes place externally. For instance, +the CHS preview link, even if your study takes place externally. For instance, although much of our peer review documentation assumes that studies are asynchronous, for a synchronous study you should expect to demonstrate your entire procedure, from scheduling all the way to debriefing. Researchers who act as reviewers will -start from a Lookit study link, sign up for a session and communicate with the +start from a CHS study link, sign up for a session and communicate with the experimenter exactly as a participant would, and attend & record a video chat session including all warmup, consent, and debrief that a participant would experience. @@ -113,11 +113,11 @@ With this feedback you can revise and improve your study, often the instructions **You should also expect to return the favor and review other labs’ studies**, as part of participating in the platform. -Step 3: Lookit submission and approval +Step 3: CHS submission and approval ====================================== When your study is ready and you’ve responded to the peer feedback, -you can :ref:`submit your study ` for Lookit approval so it can go live. +you can :ref:`submit your study ` for CHS approval so it can go live. This is the point where you'll need to have your access agreement set up. When you submit the study, you will be prompted to note any non-standard elements @@ -125,7 +125,7 @@ that require specific approval per the Terms of Use (e.g., integration of additional information about participants from another source) as well as what changes you made based on peer feedback. -At this point, Lookit staff will review your study, focusing +At this point, CHS staff will review your study, focusing primarily on the participant experience. This is also when any custom code you're using will be reviewed for security or functionality issues. @@ -136,14 +136,14 @@ To minimize the number of rounds of review needed, researchers are strongly encouraged to polish their studies as much as possible before submitting - please don't use us as a proofreading service! -.. admonition:: Outcomes of Lookit review +.. admonition:: Outcomes of CHS review In general, although we technically reserve the right not to host work at our - discretion, the outcomes of the Lookit admin review process are "accept" and + discretion, the outcomes of the CHS admin review process are "accept" and "revise and resubmit." We'll work with you to get your study ready to go. - In rare cases, a study may be fine to run on Lookit, but in the judgment of Lookit - admin staff, not a good idea to advertise publicly via Lookit. (E.g., a study that is + In rare cases, a study may be fine to run on CHS, but in the judgment of CHS + admin staff, not a good idea to advertise publicly via CHS. (E.g., a study that is ethically designed, but stands an unusual chance of putting parents off coming back for other studies.) In this case a "partial approval" - to collect data, but not make discoverable - may be the final outcome. We diff --git a/docs/source/community-study-review-basic-requirements.rst b/docs/source/community-study-review-basic-requirements.rst index f361ed70..6b038805 100644 --- a/docs/source/community-study-review-basic-requirements.rst +++ b/docs/source/community-study-review-basic-requirements.rst @@ -11,7 +11,7 @@ requirements, please ask them on Slack. In addition to the requirements listed below, you are also strongly encouraged to review the :ref:`study guidelines` -that we use for reviewing Lookit experiments. You are also welcome to give and +that we use for reviewing internal Lookit and jsPsych experiments. You are also welcome to give and solicit peer review for your studies on Slack, and we recommend that all studies do this! @@ -49,7 +49,7 @@ Terms of use ------------ Your study (including the entire experience that families go through after clicking -on the external link) must follow the `Terms of Use `__. +on the external link) must follow the `Terms of Use `__. Below are some highlights from these terms that researchers will need to be aware of: @@ -60,7 +60,7 @@ Below are some highlights from these terms that researchers will need to be awar - Researchers may not use email addresses or other participant information for any purpose other than conducting that specific study. (In fact, we recommend that you don't collect email addresses at all, and instead - :ref:`use the Lookit messaging system` to contact families!) + :ref:`use the CHS messaging system` to contact families!) .. admonition:: Lab contact databases diff --git a/docs/source/community-study-review-checklists-self.rst b/docs/source/community-study-review-checklists-self.rst index 92bc40b8..d970eeac 100644 --- a/docs/source/community-study-review-checklists-self.rst +++ b/docs/source/community-study-review-checklists-self.rst @@ -4,7 +4,7 @@ Self review checklist ===================== Here is a checklist to work through as you prepare to submit your study. -Lookit staff will look through these categories as well during internal +CHS staff will look through these categories as well during internal review, so you can avoid unnecessary rounds of review by thinking through each item before submission. @@ -79,10 +79,10 @@ Discoverable When you start a new study, we recommend making it non-discoverable while you pilot. That way you can take your time fixing any issues that come up during piloting. Once -it's discoverable, it will be listed on the Lookit "studies" page and email announcements +it's discoverable, it will be listed on the CHS "studies" page and email announcements will be sent out to all families with a child who's eligible. -- If you want your study posted on the Lookit studies page once you start collecting data, +- If you want your study posted on the CHS studies page once you start collecting data, is the “discoverable” box checked? If you don’t (because you want to pilot or recruit only from your own database), is it unchecked? @@ -94,7 +94,7 @@ Researcher contact information Exit URL -------- -- Is this someplace reasonable (e.g. https://lookit.mit.edu/studies/history)? Make sure it’s not the +- Is this someplace reasonable (e.g. https://childrenhelpingscience.com/studies/history)? Make sure it’s not the staging server/localhost/etc. Purpose / “What we’re studying” @@ -143,13 +143,13 @@ Compensation said three days, that means that you have through Monday for participants from Friday. They may be counting on the money. -Eligibility description, min/max ages, eligibility criteria expression (all self-review & Lookit review only) -------------------------------------------------------------------------------------------------------------- +Eligibility description, min/max ages, eligibility criteria expression +----------------------------------------------------------------------------- - Are any eligibility criteria beyond age either language-based (e.g., speaking English - or being bilingual) or rare (e.g., ASD)? We :ref:`generally ask that other - criteria be implemented as part of analysis, rather than preventing families from - participating.`. + or being bilingual) or rare (e.g., ASD)? We generally ask that other + criteria be :ref:`implemented as part of analysis, rather than preventing families from + participating`. - Don’t specify the age range in the criteria expression in addition to the min/max ages (it just introduces some potential for confusion if @@ -182,12 +182,12 @@ Protocol configuration - Is your study being randomized correctly? (e.g., you have the right audio and videos for the conditions they’re intended to be for) Note: - this is NOT something Lookit staff will confirm for you during + this is NOT something CHS staff will confirm for you during review; we will generally run through one random condition focusing on communication and any technical issues. - Are the audio/videos running the way you want them to? (e.g. video is located in the right place on the screen) Again, this is NOT - something Lookit staff will confirm for you as we don’t know how you + something CHS staff will confirm for you as we don’t know how you wanted them to look! - Are all stimuli hosted at URLs starting with https://, not http://? (Insecure hosts won’t be allowed for both security and performance diff --git a/docs/source/contribute-edit-docs.rst b/docs/source/contribute-edit-docs.rst index 25a905e4..fd1cdc5a 100644 --- a/docs/source/contribute-edit-docs.rst +++ b/docs/source/contribute-edit-docs.rst @@ -12,7 +12,7 @@ Advanced instructions If you want to make bigger changes, such as reorganizing content or adding new pages, it'll be easiest to work with a local copy of the documentation. -Documentation for use of the Lookit platform (what you're reading now!) lives in the `lookit-docs repo `_. You can fork this repository to create your own copy of it on Github, and then clone that fork so you have a local copy of the docs to edit in a familiar text editor. After cloning the repo, navigate to the root and run:: +Documentation for use of the CHS platform (what you're reading now!) lives in the `lookit-docs repo `_. You can fork this repository to create your own copy of it on Github, and then clone that fork so you have a local copy of the docs to edit in a familiar text editor. After cloning the repo, navigate to the root and run:: git submodule init git submodule update diff --git a/docs/source/contribute-github-issue.rst b/docs/source/contribute-github-issue.rst index 87246fa4..cc3081bb 100644 --- a/docs/source/contribute-github-issue.rst +++ b/docs/source/contribute-github-issue.rst @@ -4,12 +4,12 @@ Using Github issues ================================== -We use Github Issues to plan Lookit development. This is a central tool where we keep information about known bugs, planned new features, and what's going to be addressed when. +We use Github Issues to plan CHS development. This is a central tool where we keep information about known bugs, planned new features, and what's going to be addressed when. .. image:: _static/img/tutorial/issues_tab.png :alt: See all issues on GitHub -If you report a problem or mention an idea you have to improve Lookit through some other channel, we will probably ask that you make a Github issue so we can keep track of your idea and get feedback from any other researchers affected. +If you report a problem or mention an idea you have to improve CHS through some other channel, we will probably ask that you make a Github issue so we can keep track of your idea and get feedback from any other researchers affected. To request a feature or report a bug, first search the existing issues to see if your idea is already there. @@ -18,13 +18,15 @@ To request a feature or report a bug, first search the existing issues to see if Depending on the type of problem you are encountering or idea you have, you will want to check in one of the following repositories or "repos": -- `lookit-api `_ is the repo for the Lookit site: issues with anything to do with participant login or data, how current and past studies are displayed to participants, how you view data and manage your studies +- `lookit-api `_ is the repo for the CHS site: issues with anything to do with participant login or data, how current and past studies are displayed to participants, how you view data and manage your studies -- `ember-lookit-frameplayer `_ is the repo for the experiment components themselves: issues with how particular frames behave, frames you'd find useful, counterbalancing/condition assignment, etc. +- `ember-lookit-frameplayer `_ is the repo for the Lookit experiment runner: issues with how particular frames behave, frames you'd find useful, counterbalancing/condition assignment, etc. + +- `lookit-jspsych `_ is the repo for the jsPsych experiment runner. - `lookit-docs `_ is the repo for the documentation: anything about the docs you're reading now! -If you find a relevant issue already exists, please comment on it or add a thumbs-up reaction so Lookit staff know there's more interest! If not, click the green "New issue" button at the top right. +If you find a relevant issue already exists, please comment on it or add a thumbs-up reaction so CHS staff know there's more interest! If not, click the green "New issue" button at the top right. .. image:: _static/img/tutorial/new_issue.png :alt: Create new issue on GitHub @@ -44,4 +46,4 @@ Your issue will now have a number assigned to it and will be listed in the issue .. image:: _static/img/tutorial/issue.png :alt: Issue on GitHub -Lookit staff may respond to ask for further information, schedule it for future development, and/or wait for community feedback about the idea to gauge demand. +CHS staff may respond to ask for further information, schedule it for future development, and/or wait for community feedback about the idea to gauge demand. diff --git a/docs/source/contributor-guidelines.rst b/docs/source/contributor-guidelines.rst index 1ff4a971..42b18acf 100644 --- a/docs/source/contributor-guidelines.rst +++ b/docs/source/contributor-guidelines.rst @@ -4,13 +4,13 @@ Contributor Guidelines ================================== -Interested in helping write the code behind the Lookit platform? Thanks for supporting open source science! This page describes the process any would-be contributor should plan to use. We have included some beginner-friendly details in case you are new to open source projects. +Interested in helping write the code behind the CHS platform? Thanks for supporting open source science! This page describes the process any would-be contributor should plan to use. We have included some beginner-friendly details in case you are new to open source projects. -The content of this page applies to all three Lookit repos: ``lookit-api`` (Lookit site), ``ember-lookit-frameplayer`` (system for displaying experiments & components to use), and ``lookit-docs`` (the documentation you're reading now). +The content of this page applies to all four CHS repos: ``lookit-api`` (CHS site), ``ember-lookit-frameplayer`` (Lookit experiment runner), ``lookit-jspsych`` (jsPsych experiment runner), and ``lookit-docs`` (the documentation you're reading now). .. admonition:: Where's the code I need? - If you only want to change something about the Lookit site, without touching experiment functionality (for instance, to add a question to the demographic survey or change how studies are sorted), you will only need to run `lookit-api` and can follow the Django project installation steps. If you want to develop experiment frames or change how the experiment player works, you will need to follow the steps for local frame development, installing *both* `lookit-api` and `ember-lookit-frameplayer` and telling them how to talk to each other. Your changes, however, will likely be limited to `ember-lookit-frameplayer`. + If you only want to change something about the CHS site, without touching experiment functionality (for instance, to add a question to the demographic survey or change how studies are sorted), you will only need to run ``lookit-api`` and can follow the Django project installation steps. If you want to develop Lookit experiment frames or change how that experiment runner, you will need to follow the steps for local Lookit experiment runner development, installing *both* ``lookit-api`` and ``ember-lookit-frameplayer`` and telling them how to talk to each other. Your changes, however, will likely be limited to ``ember-lookit-frameplayer``. For the lookit-jspsych experiment runner, you will also need to install *both* ``lookit-api`` and ``lookit-jspsych``, and set up your local ``lookit-api`` to import your local ``lookit-jspsych`` packages. However, if you you want to develop your own jsPsych plugins (trials), you can do without installing any CHS code - just follow the jsPsych documentaton for `setting up your environment `_ and `developing plugins `_. Prerequisites ~~~~~~~~~~~~~~~ @@ -24,9 +24,9 @@ Getting started At a high level, we are roughly following a Forking Workflow version of Gitflow `as described here `__. -You should plan to make feature-specific branches off of the ``develop`` branch (for lookit-api, lookit-docs) or ``master`` branch (for ember-lookit-frameplayer) of a local copy of the code running on your own machine. This will keep the codebase as clean as possible. +You should plan to make feature-specific branches off of the ``develop`` branch (for ``lookit-api``, ``lookit-docs``), ``master`` branch (for ``ember-lookit-frameplayer``), or ``main`` branch (for ``lookit-jspsych``) of a local copy of the code running on your own machine. This will keep the codebase as clean as possible. -First create your own fork of lookit-api, ember-lookit-frameplayer, and/or lookit-docs. Follow the directions for installation of lookit-api or ember-lookit-frameplayer if needed. +First create your own fork of ``lookit-api``, ``ember-lookit-frameplayer``, ``lookit-jspsych``, and/or ``lookit-docs``. Follow the directions for installation of ``lookit-api`` or ``ember-lookit-frameplayer`` if needed. Ignoring some files @@ -94,9 +94,9 @@ Next, push all your local changes to your own fork. You should push your code (m Prior to finalizing your commit, make sure to clean up your code to comply with PEP8. Since both black and isort are included in our development dependencies, you should just be able to run ``isort -rc . --skip venv`` to fix your imports, and similarly ``black . --exclude=venv`` to "blacken" your changes. With both commands, replace ``venv`` with the actual name of your virtual env directory so that you don't blacken/isort your dependencies. -When your branch is ready (you've tested your changes out, and your code has comments and tests), submit a Pull Request! To do this, go to GitHub, navigate to your fork (in this case the github extension should be /your-username/lookit-api), then click ``new pull request``. Change the base to ``develop`` and the compare to ``feature/my-validation-feature``. Finally, click `Create pull request` and describe the changes you have made. Your pull request will be reviewed by Lookit staff; changes may be requested before changes are merged into the develop branch. To allow Lookit staff to add changes directly to your feature branch, follow the directions `here `_. +When your branch is ready (you've tested your changes out, and your code has comments and tests), submit a Pull Request! To do this, go to GitHub, navigate to your fork (in this case the github extension should be /your-username/lookit-api), then click ``new pull request``. Change the base to ``develop`` and the compare to ``feature/my-validation-feature``. Finally, click `Create pull request` and describe the changes you have made. Your pull request will be reviewed by CHS staff; changes may be requested before changes are merged into the develop branch. To allow CHS staff to add changes directly to your feature branch, follow the directions `here `_. -IMPORTANT: WHEN YOUR PR IS ACCEPTED, stop using your branch right away (or delete it altogether). New features (or enhanced versions of your existing feature) should be created on brand new branches (after pulling in all the fresh changes from ``develop``). +IMPORTANT: WHEN YOUR PR IS ACCEPTED, stop using your branch right away (or delete it altogether). New features (or enhanced versions of your existing feature) should be created on brand new branches based off the most up-to-date versions of the repository's 'base' branch (i.e. ``develop`` for ``lookit-api`` and ``lookit-docs``, ``master`` branch for ``ember-lookit-frameplayer``, or ``main`` branch for ``lookit-jspsych``). Writing your tests ~~~~~~~~~~~~~~~~~~~~ @@ -114,8 +114,10 @@ In ``ember-lookit-frameplayer``, you should generally add a test file under ``te To learn more about how testing is supposed to work for ``ember-lookit-frameplayer``, see https://guides.emberjs.com/v2.11.0/testing/. +For ``lookit-jspsych``, we use the `Jest framework `_ and have re-used much of the `jsPsych test configuration `_. If you add or modify any ``lookit-jspsych`` code, you will likely need to add/edit tests that live alongside the source code in the ``.spec.ts`` files. -Creating a release (ember-lookit-frameplayer) + +Creating a Lookit experiment runner release ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The ember-lookit-frameplayer repo is semantically versioned. @@ -149,4 +151,10 @@ To create a new bugfix release for an older version: 2. Apply appropriate patch and increment version on new bugfix branch. -3. Create a new release on GitHub, exactly matching new version name. Include release notes explaining what has been fixed. \ No newline at end of file +3. Create a new release on GitHub, exactly matching new version name. Include release notes explaining what has been fixed. + + +Creating a jsPsych experiment runner release +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Coming soon! \ No newline at end of file diff --git a/docs/source/faq.rst b/docs/source/faq.rst index 73435d91..abf0d0bb 100644 --- a/docs/source/faq.rst +++ b/docs/source/faq.rst @@ -1,8 +1,15 @@ FAQ ===== -What is the name of this platform? LoOkIt, Look!t, LooKit? ----------------------------------------------------------- +What is the name of this platform? Children Helping Science or Lookit? +--------------------------------------------------------------------------- + +Children Helping Science (CHS) is the name of the website, and Lookit is the name of the 'default' experiment runner. + +The whole platform was called Lookit when it was first created. In May 2023, it merged with a separate-but-similar project called Children Helping Science. Now, the platform (website) is called Children Helping Science (CHS), and the internal experiment runner is called Lookit. So if you're used to using the "Lookit" name for the whole site, you'll still see that term used to refer to a specific study type, i.e. studies that use the ember-lookit-frameplayer. You'll also see some remnants of the previous naming system in, for instance, the names of our open-source code repositories - the codebase for the CHS website is "lookit-api" and the CHS documentation is "lookit-docs". + +What is the name of the internal experiment runner? LoOkIt, Look!t, LooKit? +-------------------------------------------------------------------------------------- It is called ``Lookit``. Like the exclamation. In particular it is not called ``LookIt`` or ``Lookit!``, although the latter does sound extra @@ -18,7 +25,7 @@ Does my study need to be approved by MIT’s IRB? Nope! You need ethical approval only from your own institution. You’re responsible for getting that, although there is example language here -for common elements of Lookit studies. +for common elements of CHS studies. Separately, your lab and institution will need to sign a one-time institutional agreement with MIT. @@ -31,12 +38,9 @@ MIT Who should I list as a contact in case our contracts/sponsored programs/legal/etc. office has questions? -------------------------------------------------------------------------------------------------------- -Melissa Kline Struhl, lookit@mit.edu - -(Note: In early 2024 while Melissa is on maternity leave, these messages will still be recieved by -another member of the CHS team.) +Melissa Kline Struhl, childrenhelpingscience@gmail.com -Is there an IRB protocol at MIT that covers Lookit staff’s use of the data on the platform? +Is there an IRB protocol at MIT that covers CHS staff’s use of the data on the platform? ------------------------------------------------------------------------------------------- No, because we only use the data for a limited set of purposes that @@ -63,12 +67,12 @@ you want: - You'll likely need to write a translation file for your language for the components in the studies - e.g., consent, setup. Once that's available, you can specify a - language for your study and any hard-coded text will be translated. (See the experiment runner's :ref:`language parameter`.) + language for your study and any hard-coded text will be translated. (See the Lookit experiment runner's :ref:`language parameter` and the lookit-jsPsych locale parameters). - We haven’t currently set up to flag which language a study is in and let participants filter (or get the right version of a study) by language. So non-English studies in general will usually to be set up - as “non-discoverable” - not listed on lookit.mit.edu/studies, just + as “non-discoverable” - not listed on childrenhelpingscience.com/studies, just accessible by direct link - to avoid confusion. - You will need to handle any tech support for your participants @@ -89,8 +93,8 @@ Where are the data stored? A1: In the USA. -A2: Videos are stored primarily on Amazon S3. Lookit (including databases with -participant and session data) is hosted on Google Cloud Platform. +A2: Videos are stored primarily on Amazon S3. The CHS site, including databases with +participant and session data, is hosted on Google Cloud Platform. How is participant data secured? -------------------------------- @@ -104,17 +108,17 @@ Google infrastructure. For an extensive treatment of the security-related provisions of this cloud infrastructure, please see `Google’s white paper `__. Service Accounts are used for all services provided by the Google Cloud -Platform project instance. Permissions for access to data via the Lookit +Platform project instance. Permissions for access to data via the CHS interface and API are handled using Django Guardian according to best practices. -User passwords on Lookit are required to be 16+ characters. Participants +User passwords on CHS are required to be 16+ characters. Participants can access only their own video data; however, as researchers may be able to access video and other data from many participants in studies -they have run, access to the Experimenter section of Lookit requires +they have run, access to the Experimenter section of CHS requires two-factor authentication. Researchers are responsible for the security -of their Lookit credentials and for the security of data that they -download using Lookit, although the platform is engineered with a focus +of their CHS credentials and for the security of data that they +download using CHS, although the platform is engineered with a focus on making it more difficult to accidentally disclose sensitive information. (For instance, consent must be confirmed before session data or video are accessible; child names and birthdates are by default @@ -129,7 +133,7 @@ stored in the etcd database of a Google Kubernetes Engine instance, per the defaults provided by the platform. Video data are encrypted at rest on S3 using AES-256. -All dependencies for the Lookit-api and Ember-lookit-frameplayer +All dependencies for the website (lookit-api), Lookit experiment runner (ember-lookit-frameplayer) and jsPsych experiment runner (lookit-jspsych) repositories are continuously scanned by Github for security vulnerabilities, and the unit tests conducted as part of our CI/CD pipeline whenever code is updated cover many of the platform-specific @@ -148,9 +152,9 @@ addresses (unless participants email them). We hired an external security consulting firm to conduct detailed manual penetration and a security risk assessment prior to launch, in spring 2020. Results of this assessment are available upon request (email -lookit@mit.edu). +childrenhelpingscience@gmail.com). -Does Lookit collect IP addresses? +Does CHS collect IP addresses? --------------------------------- No. @@ -161,13 +165,13 @@ How is re-identification prevented? There are several measures in place to *discourage* re-identification, including: -- Researchers using the Lookit platform do not receive direct access to participant email addresses. They can contact participants using the Lookit interface based on the participant’s random ID, but see an email address only if a participant contacts them. -- Although each child and each family registered on Lookit is associated with a global unique random identifier, they are also associated with a different random identifier specific to each study, and the latter is the primary ID used by researchers. Per the Terms of Use, researchers may not publish the global identifiers, as these could link data across studies in ways that could lead to unanticipated re-identification potential. +- Researchers using the CHS platform do not receive direct access to participant email addresses. They can contact participants using the CHS interface based on the participant’s random ID, but see an email address only if a participant contacts them. +- Although each child and each family registered on CHS is associated with a global unique random identifier, they are also associated with a different random identifier specific to each study, and the latter is the primary ID used by researchers. Per the Terms of Use, researchers may not publish the global identifiers, as these could link data across studies in ways that could lead to unanticipated re-identification potential. - Default data downloads minimize the amount of personally identifiable information included: e.g., researchers have to specifically request columns for the child name, birthdate, parent name, etc. By default, neither the child’s birthdate nor exact age at time of participation (which could be combined with timestamps to produce a birthdate) are included, and a rounded age is provided for ease of responsibly publishing raw data. - Per the Terms of Use, no participant demographic information may be published in such a way that individual responses can be linked to participant video. However, we cannot fully *prevent* re-identification using solely -technical means, as some of the data collected on Lookit is by nature +technical means, as some of the data collected on CHS is by nature potentially identifying (e.g., video of faces). All researchers using the platform must have IRB approval for data collection, which includes assurances that they will not attempt to re-identify participants. The diff --git a/docs/source/features.rst b/docs/source/features.rst index 5a06d5c5..969e709a 100644 --- a/docs/source/features.rst +++ b/docs/source/features.rst @@ -3,8 +3,7 @@ Features ================================== - -Here are some of the features and advantages researchers using Lookit +Here are some of the features and advantages researchers using CHS benefit from: Common resources and processes @@ -28,7 +27,7 @@ Community and documentation - Active research **user community** and responsive **technical support**. -- 100% **open source**, forever. That means all of the source code that Lookit uses +- 100% **open source**, forever. That means all of the source code that CHS uses is publicly available, and you are free to extend and use it for your own purposes. @@ -48,7 +47,7 @@ Study management tools designed for developmental researchers - Built-in **consent video management**: review consent videos in the web browser and confirm that the parent provided informed consent to participate. Information about this confirmation is stored on - the Lookit server and only data from sessions with confirmed + the CHS server and only data from sessions with confirmed consent are available for viewing or download. .. image:: _static/img/wiki/consent_manager.png @@ -71,7 +70,7 @@ Study management tools designed for developmental researchers - **Contact participants** to invite them to another session of a longitudinal study, provide compensation, or ask clarifying - questions. Email records for each study are accessible on Lookit + questions. Email records for each study are accessible on CHS and you can download them for your records. - For **longitudinal studies**, determine which tasks and stimuli to use @@ -89,10 +88,10 @@ Study management tools designed for developmental researchers - **Preview** exactly how your study will work ahead of starting data collection, including collecting sample data so you can set up analysis workflows ahead of time. You control which researchers - can preview your study, and can share it by link with all Lookit + can preview your study, and can share it by link with all CHS researchers to get feedback if you choose. -- Create your own lab on Lookit to share studies among your lab +- Create your own lab on CHS to share studies among your lab members. **Easily collaborate** on studies with fine-grained permission roles for individual studies. @@ -104,14 +103,14 @@ Study management tools designed for developmental researchers participants or members of a registry) to participate based on a unique study link. -- Studies are deployed in their **own containers**, using a snapshot of - the experiment runner codebase. Updates to the code never affect +- Lookit studies are deployed in their **own containers**, using a snapshot of + the Lookit experiment runner codebase. Updates to the code never affect how your existing study works unless you choose to update! An experiment runner custom-designed for developmental paradigms ----------------------------------------------------------------- -(although we’re also interested in supporting other experiment building systems you may be familiar with!) +(Although we also support the jsPsych experiment runner, and we're interested in supporting other experiment building systems you may be familiar with!) - Use a growing library of `built-in components `__ @@ -146,15 +145,15 @@ An experiment runner custom-designed for developmental paradigms Limitations ----------------- -Lookit won’t be the right approach for all online developmental +CHS and Lookit won’t be the right approach for all online developmental research! Here are some cases where other tools will likely be a better fit: -- Medical research conducted by HIPAA-covered entities or individuals. Lookit is not +- Medical research conducted by HIPAA-covered entities or individuals. CHS is not yet HIPAA compliant. (Note that HIPAA applies based on the status of the researcher/healthcare provider - not just on the type of information collected. Academic researchers can generally collect health information without - being covered by HIPAA, and more than one hospital-based research labs have successfully negotiated data agreements allowing them to use Lookit for some types of studies.) + being covered by HIPAA, and more than one hospital-based research labs have successfully negotiated data agreements allowing them to use CHS for some types of studies.) - Short one-off survey studies where you want to minimize time required to participate, and even asking families to create a login would be diff --git a/docs/source/frame-dev-creation.rst b/docs/source/frame-dev-creation.rst index 38a03204..0df046a1 100644 --- a/docs/source/frame-dev-creation.rst +++ b/docs/source/frame-dev-creation.rst @@ -1,7 +1,7 @@ -Creating custom frames -========================== +Creating custom Lookit frames +====================================== Getting Started ~~~~~~~~~~~~~~~ diff --git a/docs/source/frame-dev-setup.rst b/docs/source/frame-dev-setup.rst index 4c22d310..0b94c93b 100644 --- a/docs/source/frame-dev-setup.rst +++ b/docs/source/frame-dev-setup.rst @@ -3,7 +3,7 @@ Setup for local development =================================== -These instructions will walk you through setting up to run Lookit locally. +These instructions will walk you through setting up to run CHS locally. Overview -------- @@ -14,18 +14,18 @@ For a full local development setup, we will need to install *both* the the Djang (``lookit-api``) and the Ember app (``ember-lookit-frameplayer``), tell them how to talk to each other, and run both of those servers locally. -- On Lookit, we will add some basic information to our superuser, and then add a child and demographic data. +- On CHS, we will add some basic information to our superuser, and then add a child and demographic data. - We then create a study locally. -- In ember-lookit-frameplayer, we'll add a token which gets added to the headers of the API requests so that Lookit knows about the logged-in user making the request. +- In ember-lookit-frameplayer, we'll add a token which gets added to the headers of the API requests so that CHS knows about the logged-in user making the request. - We can then navigate directly to the study from the Ember app to bypass the build process locally. Option 2: ~~~~~~~~~~~~~~~~ -If you are only making changes to the experiment runner (ember-lookit-frameplayer), you have the option of running only the Ember app locally, and having it talk to the Lookit staging server instead of a local Lookit server. +If you are only making changes to the Lookit experiment runner (ember-lookit-frameplayer), you have the option of running only the Ember app locally, and having it talk to the CHS staging server instead of a local CHS server. Either way, you will be able to make changes to frames locally and immediately see the results of those changes, participating in a study just as if you -were a participant on the Lookit website. You will edit the study definitions, and see +were a participant on the CHS website. You will edit the study definitions, and see the collected data, on your local instance or on the staging server, depending on the option you choose. @@ -37,7 +37,7 @@ Note: this is optional if you are only making changes to ember-lookit-frameplaye 1. Follow the instructions to install the `django app `__ locally. Run the server. -2. Navigate to https://localhost:8000/login/ to log in to Lookit. Use the superuser +2. Navigate to https://localhost:8000/login/ to log in to CHS. Use the superuser credentials created in the django installation steps, and set up 2-factor authentication so you'll be able to access the researcher and admin interfaces @@ -102,9 +102,9 @@ Ember App steps 3. Add your token and lookit-api local host address to the ember-lookit-frameplayer/.env file. This will allow your Ember app to talk - to your local API or to the Lookit staging server, depending on the option you chose + to your local API or to the CHS staging server, depending on the option you chose above. If you are using a local installation of lookit-api, insert the token you saved - earlier. If you are using the Lookit staging server, please contact the admins for + earlier. If you are using the CHS staging server, please contact the admins for an API token for your account. Your .env file will now look like this: .. code:: bash @@ -114,8 +114,8 @@ Ember App steps LOOKIT_API_KEY='Token ' LOOKIT_API_HOST='https://localhost:8000' - If you are using the Lookit staging server, this will be identical except that the - last line should be ``LOOKIT_API_HOST='https://lookit-staging.mit.edu'``. + If you are using the CHS staging server, this will be identical except that the + last line should be ``LOOKIT_API_HOST='https://babieshelpingscience.com'``. 4. Run the ember server: diff --git a/docs/source/install-ember-app.rst b/docs/source/install-ember-app.rst index c4f388f2..fd52430c 100644 --- a/docs/source/install-ember-app.rst +++ b/docs/source/install-ember-app.rst @@ -1,14 +1,14 @@ Installation: ember-lookit-frameplayer (Ember app) ================================================== -``ember-lookit-frameplayer`` is a small Ember application that allows both researchers to -preview an experiment and users to participate in an experiment. This is +``ember-lookit-frameplayer`` is the Lookit experiment runner. It is a small Ember application that allows researchers to design experiments, and then +preview or participate in the experiment. This is meant to be used in conjunction with the `Lookit API Django project `__, which contains the -Experimenter and Lookit applications. The Django application will proxy -to these Ember routes for previewing/participating in an experiment. +rest of the code for the CHS site. The Django application will proxy +to these Ember routes for previewing/participating in a Lookit experiment. -In order to run the frame player as it works on Lookit, you will need to +In order to run the Lookit experiment runner as it works on CHS, you will need to additionally install the Django app ``lookit-api`` and then follow the local frame development instructions to make sure it communicates with the Ember app. This way, for instance, an experiment frame will be able diff --git a/docs/source/other-learning-materials.rst b/docs/source/other-learning-materials.rst index d34014c1..2a1ff803 100644 --- a/docs/source/other-learning-materials.rst +++ b/docs/source/other-learning-materials.rst @@ -10,6 +10,11 @@ Learning materials Here are some slides and videos you may want to check out while getting started or when training other lab members! +.. admonition:: "Children Helping Science" vs "Lookit" + + Note that most of these materials were created prior to the merger between Children Helping Science and Lookit, and therefore use the "Lookit" term to refer to the entire site infrastructure. We now refer to the "Lookit" website/platform as "Children Helping Science" (CHS), and we still use the "Lookit" term for the internal experiment runner. + + [PDF] `Introduction to Lookit: a 20-minute overview (Jenna Croteau) `__ [Video] `CBMM Tutorial: Using Lookit to run developmental studies online (Maddie Pelz) `_ @@ -26,17 +31,17 @@ Here are some slides and videos you may want to check out while getting started [Code] `Some example FFMPEG commands (Kim Scott) `_ -If you give a presentation about using Lookit or prepare training materials for your lab, please share them here! You can :ref:`propose that change directly` or email lookit@mit.edu. +If you give a presentation about using CHS or prepare training materials for your lab, please share them here! You can :ref:`propose that change directly` or email childrenhelpingscience@gmail.com. ----------------------- Other helpful resources ----------------------- -- Most communication among Lookit researchers happens via a Slack workspace. Fill out `this form `_ to receive an invite. This is the best place to ask general questions or get tech support! +- Most communication among CHS researchers happens via a Slack workspace. Fill out `this form `_ to receive an invite. This is the best place to ask general questions or get tech support! -- The documentation for individual experiment 'frames' lives :ref:`here ` in the "Experiment runner" tab. +- The documentation for individual experiment 'frames' lives :ref:`here ` in the "Lookit experiment runner" tab. -- Running into a problem and want to check if it's a known issue, or have an idea for a handy new feature? Check out and/or add to the issues listed for the `Lookit platform `_ and for the `experiment components/player `_. Or check out `projects `_ to take a look at what's coming up in terms of development! +- Running into a problem and want to check if it's a known issue, or have an idea for a handy new feature? Check out and/or add to the issues listed for the `CHS platform `_, for the `Lookit experiment components/player `_, and for the `lookit-jspsych experiment runner `_. Or check out `projects `_ to take a look at what's coming up in terms of development! - Join the `Lookit-research email list `__ for @@ -44,14 +49,14 @@ Other helpful resources - Our March 2020 `APS Observer article `__ - which describes the current status of Lookit and how you can get your + which describes the current status of Lookit (now CHS) and how you can get your study ready! -- Lookit papers +- Lookit (now CHS) papers (`1 `__, `2 `__) -- `Lookit overview +- `Lookit (now CHS) overview video `__ - `Video data from test studies `__ @@ -66,11 +71,12 @@ Other helpful resources Codebase ----------------------- -All Lookit code is open-source (MIT License - this is a liberal +All CHS code is open-source (MIT License - this is a liberal open-source license, not related to us being at MIT) and publicly available. It will stay that way. - https://github.com/lookit/ember-lookit-frameplayer +- https://github.com/lookit/lookit-jspsych - https://github.com/lookit/lookit-api - https://github.com/lookit/lookit-docs - https://github.com/orgs/lookit/projects - software development diff --git a/docs/source/publications.rst b/docs/source/publications.rst index 6dae752c..6cfe769b 100644 --- a/docs/source/publications.rst +++ b/docs/source/publications.rst @@ -5,11 +5,11 @@ Publications .. _Publications: --------------------------------------- -Publications using Lookit data +Publications using CHS data --------------------------------------- -Here's some of the research that's come out of Lookit! There's a lot more in prep, so -if you're interested in questions like whether someone's tried X task on Lookit and how it +Here's some of the research that's come out of the CHS (formerly "Lookit") platform! There's a lot more in prep, so +if you're interested in questions like whether someone's tried X task on CHS and how it went, we recommend asking in the #researchers channel on Slack for more up-to-date info. - Beckner, A.G., Voss, A.T., Oakes, L.M., Casasola, M. (2021, April). Assessing the robustness of the mental rotation change detection procedure: the importance of task and context. In symposium: Infants’ learning about object properties and categories in diverse environments. Symposium conducted at the biennial Society for Research in Child Development (Virtual). @@ -26,7 +26,7 @@ went, we recommend asking in the #researchers channel on Slack for more up-to-da - Yoon, E. J., Frank, M. C. (2019). Preschool children's understanding of polite requests. Proceedings of the 41st Annual Conference of the Cognitive Science Society. `[pdf] `__, `[repository] `__. -If you give a presentation or publish a paper using Lookit data, please share it here! You can :ref:`propose that change directly` or email lookit@mit.edu with the citation (and a PDF or video link if you want to share it here). +If you give a presentation or publish a paper using CHS data, please share it here! You can :ref:`propose that change directly` or email childrenhelpingscience@gmail.com with the citation (and a PDF or video link if you want to share it here). ----------------------- Original Lookit papers @@ -34,7 +34,7 @@ Original Lookit papers These papers introduced a prototype of Lookit and reported on the initial test studies we ran. They may have helpful information about general considerations for unmoderated online research - e.g., do we basically see similar looking times even though the environment is noisier (spoilers: yes) and can you see where kids are looking from webcam (also yes)? -However, the platform itself has been completely re-engineered since these studies were conducted starting in 2014, so some of the details are no longer relevant - in particular, video quality is much better and we're no longer collecting variable-framerate .flv files. +However, the platform itself has been renamed to "Children Helping Science" and completely re-engineered since these studies were conducted starting in 2014, so some of the details are no longer relevant - in particular, video quality is much better and we're no longer collecting variable-framerate .flv files. - Scott, K. M. and Schulz, L. E. (2017). Lookit (part 1): a new online platform for developmental research. Open Mind 1(1):4-14. `doi:10.1162/opmi_a_00002 `__ diff --git a/docs/source/researchers-create-experiment.rst b/docs/source/researchers-create-experiment.rst index 2a997094..8f320d91 100644 --- a/docs/source/researchers-create-experiment.rst +++ b/docs/source/researchers-create-experiment.rst @@ -1,11 +1,11 @@ .. _study_protocol: -Protocol specification +Lookit protocol specification =================================== -The rest of this section is specific to internal Lookit studies that use the experiment builder (rather than sending participants to an external website.) Researchers specify how these studies work by writing a "protocol configuration" for their study. This configuration is written in JSON, which stands for JavaScript Object Notation - this is just a special text format, not code. +The rest of this section is specific to internal studies that use the Lookit experiment builder (rather than the jsPsych experiment builder or sending participants to an external website). Researchers specify how these studies work by writing a "protocol configuration" for their study. This configuration is written in JSON, which stands for JavaScript Object Notation - this is just a special text format, not code. -In the configuration, you essentially tell Lookit what sequence of "frames" to use in your study, and set all the options for those frames like what pictures or videos to show and for how long. You can see the available frames in the :ref:`experiment runner docs `. +In the configuration, you essentially tell Lookit what sequence of "frames" to use in your study, and set all the options for those frames like what pictures or videos to show and for how long. You can see the available frames in the :ref:`Lookit experiment runner docs `. Experiment structure @@ -21,7 +21,7 @@ Click on this field to bring up the experiment editor view. Here is where you define the structure of your experiment using a JSON document. (Advanced users can choose to instead provide a :ref:`'protocol generator function'`, written in Javascript, which *returns* a JSON document to use as the study protocol.) -Studies on Lookit are broken into a set of fundamental units called +Lookit studies are broken into a set of fundamental units called **frames**, which can also be thought of as “pages” of the study. A single experimental trial (e.g. looking time measurement) would generally be one frame, as are the video consent procedure and exit survey. @@ -29,7 +29,7 @@ Your study protocol will define a set of ``frames`` and also a ``sequence`` sayi order in which to use those frames. For detailed information about how to specify your study protocol, see the -:ref:`experiment runner documentation `. +:ref:`Lookit experiment runner documentation `. .. _add a protocol generator: @@ -37,7 +37,7 @@ How to add a protocol generator function ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Although your study protocol JSON can be configured to handle a wide range of common condition assignment, counterbalancing, and conditional logic schemes, in some cases it may still be more natural to programmatically generate the protocol. For examples and details about how to write a study protocol generator function, see the -:ref:`experiment runner documentation `. +:ref:`Lookit experiment runner documentation `. On your study edit form, check the "Use protocol generator (advanced)" box to use a protocol generator function in place of your study protocol: @@ -85,7 +85,7 @@ Finding and using specific frames ------------------------------------ For the most current documentation of individual frames available to -use, please see :ref:`the experiment runner documentation `. +use, please see :ref:`the Lookit experiment runner documentation `. For each frame, you will find an **example** of using it in a JSON schema; documentation of the **properties** which can be defined in the diff --git a/docs/source/researchers-day-to-day-study-operation.rst b/docs/source/researchers-day-to-day-study-operation.rst index 765e75e4..0f26c3da 100644 --- a/docs/source/researchers-day-to-day-study-operation.rst +++ b/docs/source/researchers-day-to-day-study-operation.rst @@ -12,9 +12,9 @@ Active vs. paused You can start and stop data collection independently and whenever you want. On your study detail page, go to “change state” -> “Start” or “Pause.” -When your study is active, participants can access it at the direct link shown on your study page. If the study is also discoverable, it will be listed on the Lookit "studies" page and advertised by email to eligible families. +When your study is active, participants can access it at the direct link shown on your study page. If the study is also discoverable, it will be listed on the CHS "studies" page and advertised by email to eligible families. -When your study is paused, participants can't access it even if they have the direct link. If participants follow a direct link to your study and it is paused, they will see a message `like this `__. +When your study is paused, participants can't access it even if they have the direct link. If participants follow a direct link to your study and it is paused, they will see a message `like this `__. Discoverable vs. non-discoverable ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -23,13 +23,13 @@ You can also independently switch your study between discoverable When your study is **discoverable** and active, it is: -- Listed at lookit.mit.edu/studies +- Listed at childrenhelpingscience.com/studies - Advertised to eligible families via announcement emails that include a direct link to the study, sent each day at 4am - Accessible at the direct link When your study is **non-discoverable** and active, it is: -- Not listed at lookit.mit.edu/studies +- Not listed at childrenhelpingscience.com/studies - Not advertised by announcement emails - Still accessible at the direct link! @@ -40,7 +40,7 @@ We also recommend making studies non-discoverable to do some initial piloting - Announcement emails ~~~~~~~~~~~~~~~~~~~~ -Announcement emails are sent out to families with eligible children to let them know about new studies on Lookit: +Announcement emails are sent out to families with eligible children to let them know about new studies on CHS: - Emails are only sent about studies that are **discoverable** and **active**. - Emails are only sent to families whose email preference specifies that they want to hear about new studies. @@ -54,8 +54,8 @@ Making changes to your study ---------------------------------------- If you make changes to your study - updating the fields in the -study details, the experiment runner version, etc. - your study will be -automatically “rejected” and will require review by Lookit staff again +study details, the Lookit experiment runner version, etc. - your study will be +automatically “rejected” and will require review by CHS staff again before you can make it active. This is quick but does still require manual action on our part. In general these are reviewed the same business day as submitted, often within an hour, but this is subject to whether staff is working / in a meeting / etc. Feel free to @@ -75,7 +75,7 @@ new version of Firefox, etc.!) Missing consent videos ------------------------- -If you come across a record that's missing a consent video, please (a) try refreshing the page to make sure it doesn't show up and then (b) notify Lookit staff, including the response UUID. If you're using a version of the experiment runner prior to 2.2.2 / 1.4.1 (see `releases `) then this is probably an instance of `this bug `__. We can +If you come across a record that's missing a consent video, please (a) try refreshing the page to make sure it doesn't show up and then (b) notify CHS staff, including the response UUID. If you're using a version of the Lookit experiment runner prior to 2.2.2 / 1.4.1 (see `releases `) then this is probably an instance of `this bug `__. We can fix particular instances for you but recommend updating following the instructions in `this Slack post `__. .. _compensation: @@ -84,7 +84,7 @@ Compensating participants ---------------------------------------- If you are compensating participants, in most cases you will be -messaging them through the Lookit email interface to give them gift +messaging them through the CHS email interface to give them gift cards, using the user IDs you can see in the consent manager and/or the response data. If your institution requires direct compensation or requires the collection of email addresses for other compliance reasons, you can add @@ -169,7 +169,7 @@ collected during the session. If you come across a video where you think a parent meant to consent to participate, but you do not have an adequate recording, you can email the participant to ask for confirmation. See the ‘informed consent -guidelines’ in the `Terms of Use `__ +guidelines’ in the `Terms of Use `__ for guidance. Here is an example of an email we have sent to confirm consent: @@ -190,7 +190,7 @@ consent: Sending child-related data to families ---------------------------------------- -Parents are able to review their study video in the Lookit interface. If +Parents are able to review their study video in the CHS interface. If you would like to send them additional information related to their participation, please try to do so using the “Message Participants” interface. That interface supports html but does not allow attachments. @@ -201,8 +201,8 @@ password-protected with an expiration date. If a parent requests video deleted, or you need to delete video for any other reason ------------------------------------------------------------------------------------- -Please contact Lookit staff and we will delete the video(s). You'll need to provide the +Please contact CHS staff and we will delete the video(s). You'll need to provide the response UUID. If a parent invokes GDPR specifically in their request, again please -contact Lookit (complying is straightforward but we’ll notify OGC). +contact CHS (complying is straightforward but we’ll notify OGC). diff --git a/docs/source/researchers-experiment-data.rst b/docs/source/researchers-experiment-data.rst index 8e8c3c39..2166370e 100644 --- a/docs/source/researchers-experiment-data.rst +++ b/docs/source/researchers-experiment-data.rst @@ -16,7 +16,7 @@ You can access: .. admonition:: How does it work when participants withdraw video? - If the participant selected the 'withdraw video' option in an exit-survey frame at the end of the study, all video except for the consent video is unavailable (and will be deleted from Lookit servers as well in 7 days). You will still be able to see the consent video in the consent manager. The fact that video has been withdrawn is included in the response data. + If the participant selected the 'withdraw video' option in an exit-survey frame at the end of the study, all video except for the consent video is unavailable (and will be deleted from CHS servers as well in 7 days). You will still be able to see the consent video in the consent manager. The fact that video has been withdrawn is included in the response data. There is a potential rare edge case where you access video while the participant is still doing the study, and then they withdraw, so you should still verify that none of your participants have withdrawn video. @@ -62,7 +62,7 @@ There are "global IDs" you can download for **account**, **child**, and **demogr Why two different IDs? ~~~~~~~~~~~~~~~~~~~~~~~ -Lookit participants may take part in studies from a variety of labs. This means that if researchers directly use the unique database identifiers ("global IDs") for accounts, children, and demographic data snapshots, different labs will be using the same identifiers for the same children. That's important to allow collaboration in cases where you have IRB approval to combine data from different studies, but it also means that if you and another lab both published those global IDs, someone else could come along and link data from a participant who did both studies. Usually this would be ok, but in some cases information that wasn't sensitive on its own in either study could be combined to produce more sensitive or identifying information. So we provide "hashed IDs" which are study-specific and can be published. +CHS participants may take part in studies from a variety of labs. This means that if researchers directly use the unique database identifiers ("global IDs") for accounts, children, and demographic data snapshots, different labs will be using the same identifiers for the same children. That's important to allow collaboration in cases where you have IRB approval to combine data from different studies, but it also means that if you and another lab both published those global IDs, someone else could come along and link data from a participant who did both studies. Usually this would be ok, but in some cases information that wasn't sensitive on its own in either study could be combined to produce more sensitive or identifying information. So we provide "hashed IDs" which are study-specific and can be published. What if we do need to link participants across studies and want to publish the data? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -78,7 +78,7 @@ If you do this and come up with scripts that help with the workflow, please shar What are the regular IDs? ~~~~~~~~~~~~~~~~~~~~~~~~~~ -The regular hashed IDs are six-character strings (like `6RYE3U`) that uniquely identify an account, child, or demographic snapshot **within a particular study**. The same child will always have the same hashed ID within a particular Lookit study, but that child would have a different hashed ID in a different study, even one you were running. +The regular hashed IDs are six-character strings (like `6RYE3U`) that uniquely identify an account, child, or demographic snapshot **within a particular study**. The same child will always have the same hashed ID within a particular CHS study, but that child would have a different hashed ID in a different study, even one you were running. These hashed IDs are shown in the consent manager, individual responses, all responses, and email participant views, and may be called simply IDs. @@ -109,7 +109,7 @@ To limit the potential for accidental disclosure of identifying information abou .. admonition:: What can and can't I publish? - The main items you need to avoid publishing are global IDs, birthdates, names, and demographic survey responses if they can be linked to video also published. For convenience, we note specific fields that must be redacted for publication in the CSV data dictionaries. However, if any of this is unfamiliar, please review the Lookit `Terms and Conditions `_! + The main items you need to avoid publishing are global IDs, birthdates, names, and demographic survey responses if they can be linked to video also published. For convenience, we note specific fields that must be redacted for publication in the CSV data dictionaries. However, if any of this is unfamiliar, please review the CHS `Terms and Conditions `_! On the left are options for downloading information about the age of the participant. You can choose to download actual birthdates, exact ages in days, and/or a rounded age. The rounded age is rounded to the nearest 10 days for children under 365 days and to the nearest 30 days after that. @@ -354,7 +354,7 @@ The ``response`` data contains information concerning this particular session: w - **databrary**: Whether the parent agreed to share video data on Databrary - ``yes`` or ``no``. If missing, you must treat the video as if ``no`` were selected. If ``yes``, the video privacy selections also apply to authorized Databrary users. - **is_preview**: A ``true``/ ``false`` flag indicating whether or not this response was generated by a researcher previewing the experiment. Preview data should not be used in any actual analyses. - **sequence**: A list containing the sequence of **frames** the subject actually saw (after running randomization, etc.). If the participant ended the study early, then this list will not contain all the frames in the study. The frame names follow the pattern ``-``, where ```` is the order in the overall sequence where this **frame** appeared, and ```` is the identifier of the frame as defined in the ‘frames’ property of the experiment structure. -- **conditions**: An object containing information about conditions to which the subject was assigned in any frames that do randomization (choice frames). Keys are in the format ``-`` corresponds with the ```` from the ‘sequence’ of the *original* experiment structure, and the ```` again corresponds with the identifier of the frame as defined in the ‘frames’ property of the experiment structure. Data will be stored in conditions for the *first* frame created by a randomizer (top-level only for now, i.e. not from nested randomizers). Values are objects containing mappings from condition names to their values for this session. The data stored by a particular randomizer can be found under ``data collected`` in the :ref:`experiment runner docs ` +- **conditions**: An object containing information about conditions to which the subject was assigned in any frames that do randomization (choice frames). Keys are in the format ``-`` corresponds with the ```` from the ‘sequence’ of the *original* experiment structure, and the ```` again corresponds with the identifier of the frame as defined in the ‘frames’ property of the experiment structure. Data will be stored in conditions for the *first* frame created by a randomizer (top-level only for now, i.e. not from nested randomizers). Values are objects containing mappings from condition names to their values for this session. The data stored by a particular randomizer can be found under ``data collected`` in the :ref:`Lookit experiment runner docs ` The ``consent`` key contains information about the consent ruling for this study session: diff --git a/docs/source/researchers-lag-issues.rst b/docs/source/researchers-lag-issues.rst index 80a94caa..394e4da3 100644 --- a/docs/source/researchers-lag-issues.rst +++ b/docs/source/researchers-lag-issues.rst @@ -1,8 +1,8 @@ ################################## -Timing accuracy on Lookit +Timing accuracy in Lookit studies ################################## -Lookit data provides detailed timing information which allows a researcher to +Data from Lookit experiments provides detailed timing information which allows a researcher to identify the part of each video recording that corresponds to a set of stimuli being displayed to a participant. @@ -17,7 +17,7 @@ These differences can have two kinds of impacts on the data that is collected: How much noise or lag you can tolerate is up to your sample size and study. The larger your sample size, the less likely that noise will obscure the effect you are interested in. The more precise your study design is in its timing, the smaller a lag may be a problem for your analysis. -Thanks to recent work by Lookit researchers, we have some more specific information about how Lookit studies using audio & video stimuli (i.e. vs. text, image, etc.) are affected by lag. **In particular, studies using video stimuli will tend to have a 150-450ms lag, i.e. appearing to the participant later than the Lookit frame data will indicate.** +Thanks to recent work by CHS researchers, we have some more specific information about how Lookit studies using audio & video stimuli (i.e. vs. text, image, etc.) are affected by lag. **In particular, studies using video stimuli will tend to have a 150-450ms lag, i.e. appearing to the participant later than the Lookit frame data will indicate.** ================================================ Descriptions of lag with video stimuli @@ -26,9 +26,9 @@ Descriptions of lag with video stimuli This section reports findings by Aaron Becker, Christian Harms, Lisa Oakes, Yi Lin, and Michaela DeBolt. -In response to a Slack conversation with Yi Lin and Jenny Wang, initiated by Yi on 9/2/2021, the Oakes lab did extensive testing of the timing of stimulus onset and compared when the stimulus actually started compared to when the frame data in lookit indicated that the stimulus appeared. +In response to a Slack conversation with Yi Lin and Jenny Wang, initiated by Yi on 9/2/2021, the Oakes lab did extensive testing of the timing of stimulus onset and compared when the stimulus actually started compared to when the frame data in Lookit indicated that the stimulus appeared. -They tested this by running their experiments in Lookit and holding a mirror up to the camera. Thus the video recorded during the lookit session was the reflection of what appeared on the monitor, rather than the infants’ face (thanks to Yi Lin for pointing out this way of testing). +They tested this by running their experiments in Lookit and holding a mirror up to the camera. Thus the video recorded during the Lookit session was the reflection of what appeared on the monitor, rather than the infants’ face (thanks to Yi Lin for pointing out this way of testing). The team tested several browsers, internet speeds, and experiments and observed that there is a consistent discrepancy of between 150 and 450 ms on every trial, including the calibration trials and the experimental trials. @@ -48,7 +48,7 @@ This github issue documents the observed problem in more detail. One consequence Affected browsers and experiment frames ================================================ -The two officially supported browsers for Lookit are Chrome and Firefox. In the testing above, Chrome was generally found to have smaller lags than Firefox. +The two officially supported browsers for Lookit studies are Chrome and Firefox. In the testing above, Chrome was generally found to have smaller lags than Firefox. The following experiment frames are known to be impacted by one of these issues: diff --git a/docs/source/researchers-log-in.rst b/docs/source/researchers-log-in.rst index 51da0ff3..cd3c24c1 100644 --- a/docs/source/researchers-log-in.rst +++ b/docs/source/researchers-log-in.rst @@ -7,7 +7,7 @@ Registration and login Logging in -------------------------------------- -1. Click "Login" from the `Lookit home page `_. Both researchers and participants log in from this form. +1. Click "Login" from the `CHS home page `_. Both researchers and participants log in from this form. .. image:: _static/img/login/login_homepage_link.png :alt: Link on homepage to log in @@ -25,18 +25,18 @@ Logging in If you created your account via OSF (i.e. before August 2020) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - The first time you log in to Lookit after OSF login is unavailable, you will need to + The first time you log in to CHS after OSF login is unavailable, you will need to click "Forgot password?" to request a password reset link. .. image:: _static/img/login/login_password_reset_link.png :alt: Password reset link on login view - Follow the link you receive by email to set a new Lookit password for your account: + Follow the link you receive by email to set a new password for your account: .. image:: _static/img/login/login_password_reset.png :alt: Password reset form - When you log in again, you will be redirected to two-factor authentication setup so you can use the researcher section of Lookit! + When you log in again, you will be redirected to two-factor authentication setup so you can use the researcher section of CHS! .. image:: _static/img/login/login_previous_account_redirect.png :alt: Prompt to set up 2FA for previous account @@ -45,32 +45,32 @@ If you created your account via OSF (i.e. before August 2020) Creating a researcher account -------------------------------------- -1. Go to `the researcher registration form `_. (You can also get here from the `Lookit home page `_ by clicking "Sign up" and then following the link to register as a researcher instead of a participant.) +1. Go to `the researcher registration form `_. (You can also get here from the `CHS home page `_ by clicking "Sign up" and then following the link to register as a researcher instead of a participant.) .. image:: _static/img/login/login_researcher_registration.png :alt: Researcher registration screen 2. Fill out the form and click "create account." -3. You'll be taken to a page like this to set up two-factor authentication (2FA), which you'll need in order to access the researcher section of Lookit. If you haven't already, download the Google Authenticator app on your phone. Then follow the directions on this page to activate 2FA for your account. +3. You'll be taken to a page like this to set up two-factor authentication (2FA), which you'll need in order to access the researcher section of CHS. If you haven't already, download the Google Authenticator app on your phone. Then follow the directions on this page to activate 2FA for your account. .. image:: _static/img/login/login_2fa_setup.png :alt: Two-factor authentication setup screen -4. You're logged in! You should be redirected to a page of studies like below. You will likely want to create a new lab or request to join a lab (see :ref:`Labs on Lookit`) so you can share your studies within a lab, but you can go ahead and get started by creating studies in the "Sandbox lab." +4. You're logged in! You should be redirected to a page of studies like below. You will likely want to create a new lab or request to join a lab (see :ref:`Labs on CHS`) so you can share your studies within a lab, but you can go ahead and get started by creating studies in the "Sandbox lab." .. image:: _static/img/login/login_success_redirect.png - :alt: Lookit studies page visible after researcher login + :alt: CHS studies page visible after researcher login Creating a participant account -------------------------------------- -You do not need to make a separate participant account in order to participate with your own kids: you can log in as a researcher, then go back to Lookit and you will still be logged in. However, if you prefer to have a separate participant account, you can do so by logging out and clicking the "Sign up" button. You'll need to use a different email address from the one your researcher account uses. +You do not need to make a separate participant account in order to participate with your own kids: you can log in as a researcher, then go back to CHS and you will still be logged in. However, if you prefer to have a separate participant account, you can do so by logging out and clicking the "Sign up" button. You'll need to use a different email address from the one your researcher account uses. -1. Click "Sign up" from the `Lookit home page `_. +1. Click "Sign up" from the `CHS home page `_. .. image:: _static/img/login/login_homepage_registration_link.png - :alt: Sign up link on Lookit homepage + :alt: Sign up link on CHS homepage 2. Fill out the form and click "create account." @@ -79,22 +79,22 @@ You do not need to make a separate participant account in order to participate w .. _staging server: -The Lookit staging server +The CHS staging server ------------------------------------------------ -The Lookit "staging server" is a separate instance of the platform from the "production server" where actual studies take place. This is a sandbox environment where we try out new features, without having to worry about messing with actual data or inconveniencing users if we break something. +The Children Helping Science "staging server" is a separate instance of the platform from the "production server" where actual studies take place. This is a sandbox environment where we try out new features, without having to worry about messing with actual data or inconveniencing users if we break something. -The staging server is located at ``_. +The staging server is located at ``_. -You can register as a researcher at ``_ or as a participant by clicking "Sign up". Logging in works just the same way as at lookit.mit.edu, but accounts are separate. (Your staging login won't work on regular or "production" Lookit, and vice versa.) +You can register as a researcher at ``_ or as a participant by clicking "Sign up". Logging in works just the same way as at childrenhelpingscience.com, but accounts are separate. (Your staging login won't work on regular or "production" CHS, and vice versa.) There is **absolutely no actual data collection allowed** on the staging server for security reasons. You might use the staging server if: -* You want to have a totally separate "sandbox" version of your lab on Lookit to use for training new lab members, study development, demoing something, etc. without any concern that you could possibly disclose participant information +* You want to have a totally separate "sandbox" version of your lab on CHS to use for training new lab members, study development, demoing something, etc. without any concern that you could possibly disclose participant information * You're helping to try out a new feature before it's added to production -* You're making especially heavy use of Lookit while developing your own studies, working on an adapter so you can hook up another type of experiment runner, etc. - to avoid unusual load on the production server we might ask you to do your development using staging. +* You're making especially heavy use of CHS while developing your own studies, working on an adapter so you can hook up another type of experiment runner, etc. - to avoid unusual load on the production server we might ask you to do your development using staging. Managing 2FA credentials @@ -105,16 +105,16 @@ If needed, you can disable two-factor authentication on your account (for exampl .. image:: _static/img/login/login_researcher_manage_account.png :alt: Link to manage account from experimenter -From the account page, you can change your email address or password and enable or disable 2FA. Note that you cannot use the Experimenter section of Lookit without 2FA enabled. +From the account page, you can change your email address or password and enable or disable 2FA. Note that you cannot use the Experimenter section of CHS without 2FA enabled. .. image:: _static/img/login/login_2fa_management.png :alt: 2FA management on login page -Logging in when running Lookit locally for development +Logging in when running CHS locally for development ------------------------------------------------------- -If running Lookit locally, you can log in or create accounts as described above, substituting ``https://localhost:8000`` for ``https://lookit.mit.edu``. (You no longer need to authenticate through the admin interface with any experimenter accounts.) +If running CHS locally, you can log in or create accounts as described above, substituting ``https://localhost:8000`` for ``https://childrenhelpingscience.com``. (You no longer need to authenticate through the admin interface with any experimenter accounts.) Troubleshooting @@ -123,7 +123,7 @@ Troubleshooting I'm trying to log in but it says my account is "inactive" ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Most likely you accidentally tried to create a researcher account on Lookit before we launched, and it was inactivated. Please reach out in the #tech-support channel on Slack and we can fix it. +Most likely you accidentally tried to create a researcher account on CHS (Lookit) before we launched, and it was inactivated. Please reach out in the #tech-support channel on Slack and we can fix it. I can't register as a researcher because I already have a participant account ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -139,7 +139,7 @@ I'm being prompted for a one-time password (OTP) but I don't have that set up .. image:: _static/img/login/login_2fa_error.png :alt: 2FA error on researcher page -If you are seeing a message like this when trying to access the researcher side of Lookit, click "My Account" at the top right. From there you should be able to enable 2FA or complete setup. +If you are seeing a message like this when trying to access the researcher side of CHS, click "My Account" at the top right. From there you should be able to enable 2FA or complete setup. .. image:: _static/img/login/login_2fa_disabled.png :alt: Account management page when 2FA is disabled @@ -157,9 +157,9 @@ My OTP codes don't work First check that: -- The email address shown in your authenticator app matches the email address you're trying to log in as. If you created multiple accounts on Lookit, you may have replaced the OTP entry for one with the other. If that's the case, contact Lookit staff (#tech-support channel on Slack) for assistance. +- The email address shown in your authenticator app matches the email address you're trying to log in as. If you created multiple accounts on CHS, you may have replaced the OTP entry for one with the other. If that's the case, contact CHS staff (#tech-support channel on Slack) for assistance. - You are entering the code within the 30-second window. If using Duo, some users have found the timing is more finicky and they need to enter the code in the first 10 seconds or so. -- The OTP entry says "Lookit-production" if you're using lookit.mit.edu, and "Lookit-staging" if you're using lookit-staging.mit.edu. +- The OTP entry says "Lookit-production" if you're using childrenhelpingscience.com, and "Lookit-staging" if you're using babieshelpingscience.com. - The timing of your Authenticator app is accurate and your phone's time is accurate. If you're using Google Authenticator, you can go to the three dots in the top right corner -> Settings -> Time correction for codes -> Sync now to ensure the timing is correct. I switched to a new phone and can't get my OTP code @@ -168,11 +168,11 @@ I switched to a new phone and can't get my OTP code If you still have access to your old phone: 1. Log in to your account and enter the OTP code using your old phone. -2. Click "My account" or go to ``_ and scroll down to "Manage Two-Factor Authentication." +2. Click "My account" or go to ``_ and scroll down to "Manage Two-Factor Authentication." 3. Enter your OTP from the old phone to disable 2FA temporarily. 4. From "Manage Two-Factor Authentication," turn 2FA back on using your new phone. -If you do not have access to your old phone (e.g., it was destroyed and that's why you're switching): Please contact Lookit staff (#tech-support channel on Slack) for assistance. For security reasons, there isn't a way to disable or reset your two-factor authentication unless you are already logged in using a one-time password. Depending on whether you already have access to participant data, we will reset it for you or ask for verification of your identity first. +If you do not have access to your old phone (e.g., it was destroyed and that's why you're switching): Please contact CHS staff (#tech-support channel on Slack) for assistance. For security reasons, there isn't a way to disable or reset your two-factor authentication unless you are already logged in using a one-time password. Depending on whether you already have access to participant data, we will reset it for you or ask for verification of your identity first. diff --git a/docs/source/researchers-manage-consent.rst b/docs/source/researchers-manage-consent.rst index 603bd38c..ef528a5f 100644 --- a/docs/source/researchers-manage-consent.rst +++ b/docs/source/researchers-manage-consent.rst @@ -6,11 +6,11 @@ Coding consent Overview =========== -At the start of a Lookit study, the parent is asked to provide a verbal statement of informed consent. Unlike in the lab (or at least to a greater extent), it is technically possible for you to end up collecting data from a parent who did NOT consent to participate - e.g., someone idly clicking through who may not understand that this is a research study to do with a child. +At the start of an internal study on Children Helping Science, the parent is asked to provide a verbal statement of informed consent. Unlike in the lab (or at least to a greater extent), it is technically possible for you to end up collecting data from a parent who did NOT consent to participate - e.g., someone idly clicking through who may not understand that this is a research study to do with a child. -For this reason it is critical that you confirm informed consent before using any data from a response! This is baked into the Lookit experimenter interface: you actually do not receive access to responses, or to the associated child, account, or demographic data, until you confirm consent using the consent manager. +For this reason it is critical that you confirm informed consent before using any data from a response! This is baked into the CHS experimenter interface: you actually do not receive access to responses, or to the associated child, account, or demographic data, until you confirm consent using the consent manager. -Responses to your Lookit study will first appear in your study's 'Consent Manager' page, and they start out with a consent status of 'Pending'. Then a researcher working on this study can either 'Approve' or 'Reject' the consent video. Responses with approved consent become available to the researcher through the study's 'Study Responses' page. +Responses to your study will first appear in your study's 'Consent Manager' page, and they start out with a consent status of 'Pending'. Then a researcher working on this study can either 'Approve' or 'Reject' the consent video. Responses with approved consent become available to the researcher through the study's 'Study Responses' page. ============================== Managing consent rulings @@ -42,7 +42,7 @@ When you click on any response in the list to the left, more information about t .. tip:: It is possible, although rare, for there to be multiple consent videos associated with a single response. This would occur, for instance, when researchers are collecting both parental consent and child assent videos, which would be judged together. In these cases, you will see more than one number inside the arrow buttons below the video, and you can use the buttons to navigate across all consent videos for that session. Even with multiple videos, there will only be one comment allowed per session (since the comment box is intended for notes about the overall decision, rather than notes on individual videos). -For each response session in your list, you should watch the video and decide whether it shows informed consent. You can choose to 'Accept' or 'Reject' a response by changing the value in that response's dropdown box. You can also add a comment about that response or consent decision, if desired, to keep track of any additional information. You can enter a comment without changing the consent ruling (e.g., to say "Emailed this family to confirm consent"). In general, you should 'accept' consent only when the consent video shows an adult reading the consent statement audibly (or signing in ASL), but see the `Terms of Use `_ for details (for instance, you may be able to contact a family to confirm consent by email in some cases). +For each response session in your list, you should watch the video and decide whether it shows informed consent. You can choose to 'Accept' or 'Reject' a response by changing the value in that response's dropdown box. You can also add a comment about that response or consent decision, if desired, to keep track of any additional information. You can enter a comment without changing the consent ruling (e.g., to say "Emailed this family to confirm consent"). In general, you should 'accept' consent only when the consent video shows an adult reading the consent statement audibly (or signing in ASL), but see the `Terms of Use `_ for details (for instance, you may be able to contact a family to confirm consent by email in some cases). .. image:: _static/img/consent_manager_change_ruling.png :alt: Response list showing a comment and the dropdown consent ruling options @@ -59,9 +59,9 @@ The most recent consent ruling, the time of that ruling, any comment, and the na Managing responses that use survey consent ---------------------------------------------- -In general, all studies on CHS should use video recordings for consent. However there may be studies that collect consent through survey-style questions - for instance, in rare cases when, for legal reasons, it is not possible to collect video consent. In internal Lookit studies, this is done using the ``survey-consent`` frame. +In general, all studies on CHS should use video recordings for consent. However there may be studies that collect consent through survey-style questions - for instance, in rare cases when, for legal reasons, it is not possible to collect video consent. In internal Lookit and jsPsych studies, this is done using the ``survey-consent`` frame (Lookit) and the ``chsRecord.VideoConsentPlugin`` (jsPsych). -When a study uses survey questions for consent, researchers are still responsible for verifying that the family has actually consented by checking their answers to the consent question(s). On the Consent Manager page, any sessions in the response list that contain a ``survey-consent`` frame will be flagged with "[Survey consent]". When such a response is selected, a warning message will also appear at the top of the Video area. Depending on the study, there may or may not also be a consent video associated with the response session; if so then it will appear in the Video area, otherwise there will another warning message saying "No video found for this response". +When a study uses survey questions for consent, researchers are still responsible for verifying that the family has actually consented by checking their answers to the consent question(s). On the Consent Manager page, any sessions in the response list that contain a ``survey-consent`` frame (Lookit) or ``chsRecord.VideoConsentPlugin`` (jsPsych) will be flagged with "[Survey consent]". When such a response is selected, a warning message will also appear at the top of the Video area. Depending on the study, there may or may not also be a consent video associated with the response session; if so then it will appear in the Video area, otherwise there will another warning message saying "No video found for this response". .. image:: _static/img/consent_manager_response_selected_survey_consent_no_video.png :alt: Response that contains survey consent questions and no video @@ -98,7 +98,7 @@ A summary of responses is shown to the right of the consent manager, providing s Withdrawn responses -------------------- -If a parent chooses to withdraw video data at the end of the study, that will be noted in the list item for the response (before the comment it will say 'Withdrawn' and the response will be crossed out). All video data beyond consent will be inaccessible to researchers, and it will be deleted automatically from Lookit servers after seven days. +If a parent chooses to withdraw video data at the end of the study, that will be noted in the list item for the response (before the comment it will say 'Withdrawn' and the response will be crossed out). All video data beyond consent will be inaccessible to researchers, and it will be deleted automatically from CHS servers after seven days. However, you are still able to make a consent ruling about the consent video; this will still impact access to the remaining non-video response data as well as associated child, demographic, and account data. diff --git a/docs/source/researchers-manage-org.rst b/docs/source/researchers-manage-org.rst index c27b8b5a..cd1a501d 100644 --- a/docs/source/researchers-manage-org.rst +++ b/docs/source/researchers-manage-org.rst @@ -2,16 +2,16 @@ .. _labs: ######################################################## -Labs on Lookit +Labs on CHS ######################################################## -Each study on Lookit is associated with a lab. Some basic information about each lab is -stored on Lookit, like the lab's name, a contact email, and whether the lab has been -approved to use Lookit to collect data. In the future, this information will be used to +Each study on Children Helping Science is associated with a lab. Some basic information about each lab is +stored on CHS, like the lab's name, a contact email, and whether the lab has been +approved to use CHS to collect data. In the future, this information will be used to create a display page where participants can learn more about your lab, but for now, it is only visible to researchers. -Researcher accounts on Lookit can be affiliated with any number of labs. Only a researcher who +Researcher accounts on CHS can be affiliated with any number of labs. Only a researcher who is part of a lab can be granted permissions to studies that lab is running. Researchers can also be granted some permissions for the lab as a whole: for instance, to edit the lab description, manage membership, or see all studies the lab is running. @@ -20,12 +20,12 @@ description, manage membership, or see all studies the lab is running. The Sandbox and Demo labs -------------------------------------------------------- -There are two "special" labs that everyone on Lookit is added to automatically. When you log into Lookit for the first time, if you click on "Manage Labs," you should see something like this: +There are two "special" labs that everyone on CHS is added to automatically. When you log into CHS for the first time, if you click on "Manage Labs," you should see something like this: .. image:: _static/img/labs/initial_lab_list.png :alt: Initial list of labs showing Sandbox lab and Demo lab -You are added as a "Guest" in the **Sandbox lab.** This means you can create new studies associated with the Sandbox lab, and you can be given permissions to other studies associated with the Sandbox lab. However, you can't change the lab or see all the studies it is running. If you're working through the tutorial or trying out Lookit to see how it works, you can work on studies in the Sandbox lab and not have to create your own lab. +You are added as a "Guest" in the **Sandbox lab.** This means you can create new studies associated with the Sandbox lab, and you can be given permissions to other studies associated with the Sandbox lab. However, you can't change the lab or see all the studies it is running. If you're working through the tutorial or trying out CHS to see how it works, you can work on studies in the Sandbox lab and not have to create your own lab. You are added with "View" permissions in the **Demo lab.** This means you can see and preview all of the studies in the Demo lab, but not create any new studies in this lab. We keep some example studies in the Demo lab so that everyone can see them. You can clone any of the Demo lab studies into the Sandbox lab or your own lab so you can edit a copy. @@ -33,13 +33,13 @@ You are added with "View" permissions in the **Demo lab.** This means you can se Creating a new lab -------------------------------------------------------- -From the "Manage Labs" page, click the green "Create Lab" button. This will take you to a form where you can enter the information about your lab. When you click "Submit," the lab will be created, with you as an admin, and Lookit staff will be notified of the new lab. You will be able to see your new lab in your list of labs. +From the "Manage Labs" page, click the green "Create Lab" button. This will take you to a form where you can enter the information about your lab. When you click "Submit," the lab will be created, with you as an admin, and CHS staff will be notified of the new lab. You will be able to see your new lab in your list of labs. -------------------------------------------------------- Approval to test -------------------------------------------------------- -Once Lookit staff receive a signed institutional agreement for your lab to use Lookit, and +Once CHS staff receive a signed institutional agreement for your lab to use CHS, and at least one of your lab members has completed the Terms of Use quiz, your lab will be changed to "approved to test." Lab admins will be notified by email when the lab is approved to test. You can also see whether a lab is currently approved to test by clicking on it in the list of labs to view more details: @@ -55,7 +55,7 @@ Individual studies will still require review - the lab being approved to test is Joining an existing lab -------------------------------------------------------- -If your lab is already on Lookit, you can find it and join it. Go to "Manage Labs" and click the "All" tab. Then you can search by name, PI name, or institution to find the lab. Click "request to join" next to the lab you would like to join. (You can also click on the lab to get more details first!) +If your lab is already on CHS, you can find it and join it. Go to "Manage Labs" and click the "All" tab. Then you can search by name, PI name, or institution to find the lab. Click "request to join" next to the lab you would like to join. (You can also click on the lab to get more details first!) .. image:: _static/img/labs/lab_list.png :alt: List of lab search results with request-to-join buttons @@ -90,7 +90,7 @@ By design, there is no way to give anyone access to participant data for all stu Lab's custom URL -------------------------------------------------------- -Each lab has its own page on Lookit. This provides a convenient URL for you to distribute and allows families to easily find all of your lab's active studies. To find the current custom URL for your lab, go to "Manage Labs" and select your lab. You will find the link to the lab's page under "Custom URL". +Each lab has its own page on CHS. This provides a convenient URL for you to distribute and allows families to easily find all of your lab's active studies. To find the current custom URL for your lab, go to "Manage Labs" and select your lab. You will find the link to the lab's page under "Custom URL". To change your lab's URL, click "Edit lab" in the top right of your lab's information page. Update the "Custom URL" to suit your lab's preferences. diff --git a/docs/source/researchers-prep-stimuli.rst b/docs/source/researchers-prep-stimuli.rst index 6db75dbb..e538479f 100644 --- a/docs/source/researchers-prep-stimuli.rst +++ b/docs/source/researchers-prep-stimuli.rst @@ -164,7 +164,7 @@ Tips and tricks (advanced) Setting up a CDN (optional) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -If you are very concerned with optimizing speed of delivery of your stimuli for users worldwide, best practice is to use a Content Delivery Network (CDN). You can read a description of what this is and when it might be helpful `here `__. This is unlikely to be necessary for most Lookit researchers, but if you do choose to set one up, it's cheap and reasonably straightforward. One option we have used successfully is Amazon CloudFront. +If you are very concerned with optimizing speed of delivery of your stimuli for users worldwide, best practice is to use a Content Delivery Network (CDN). You can read a description of what this is and when it might be helpful `here `__. This is unlikely to be necessary for most CHS researchers, but if you do choose to set one up, it's cheap and reasonably straightforward. One option we have used successfully is Amazon CloudFront. Making dummy stimuli ~~~~~~~~~~~~~~~~~~~~~~ diff --git a/docs/source/researchers-set-study-fields.rst b/docs/source/researchers-set-study-fields.rst index 790b2910..5e38c108 100644 --- a/docs/source/researchers-set-study-fields.rst +++ b/docs/source/researchers-set-study-fields.rst @@ -4,7 +4,7 @@ Setting study details ################################## -When creating or editing a study, you can set the value of the following fields. Below is more information about each field. This form is used for both internal Lookit studies and both synchronous and asychronous external studies. +When creating or editing a study, you can set the value of the following fields. Below is more information about each field. This form is used for both Lookit and jsPsych internal studies, and both synchronous and asychronous external studies. ============================= @@ -30,20 +30,20 @@ Setting your study's priority value to 99 (the default) will mean it is shown *f ============================= External ============================= -Will you be providing a link to a study (or study scheduling page) rather than using the Lookit experiment builder? If so, then check this box to show the form fields that are relevant to these types of studies, such as the study/scheduling link (see :ref:`study-url`). Checking this box will also hide fields of this form that are only applicable for internal studies. Leave this box unchecked if you are building a protocol specification to be conducted inside the Lookit architecture (i.e. an 'internal' study; see `Building an Experiment`_ for more information). +Will you be providing a link to a study (or study scheduling page) rather than using the Lookit or jsPsych experiment builders? If so, then check this box to show the form fields that are relevant to these types of studies, such as the study/scheduling link (see :ref:`study-url`). Checking this box will also hide fields of this form that are only applicable for internal Lookit/jsPsych studies. Leave this box unchecked if you are building an experiment to be conducted inside the CHS architecture (i.e. an 'internal' study; see `Building an Experiment`_ for more information). ============================= Scheduled ============================= -The Lookit experiment builder doesn't have the capacity for video chat studies, so this box is greyed out unless the External box is checked. If you are running an external study, leave this box unchecked if participants will click a link and participate in a study right away, without any back-and-forth or live interaction with a researcher. Check this box if participants will wind up scheduling and attending a video chat with a researcher. +The internal experiment builders, Lookit and jsPsych, don't have the capacity for video chat studies, so this box is greyed out unless the External box is checked. If you are running an external study, leave this box unchecked if participants will click a link and participate in a study right away, without any back-and-forth or live interaction with a researcher. Check this box if participants will wind up scheduling and attending a video chat with a researcher. .. _discoverability: ============================= Discoverable ============================= -Do you want this study to be listed on the Lookit studies page when it's active, and -eligible participants in the Lookit database to receive email invitations to participate? If so, check this box to make the study discoverable. Email invitations will go out to families with eligible children at a rate of up to 50 emails per day as long as the study is active and discoverable. +Do you want this study to be listed on the CHS studies page when it's active, and +eligible participants in the CHS database to receive email invitations to participate? If so, check this box to make the study discoverable. Email invitations will go out to families with eligible children at a rate of up to 50 emails per day as long as the study is active and discoverable. If the box is unchecked, the study will be 'non-discoverable' and participants will only be able to get to it by following a direct link with your study ID. This may be helpful if, for instance, you want to run a follow-up study (with in-lab or online participants) and want to send the link to a limited number of people, or if your inclusion criteria are very limited (e.g., a rare genetic disorder) and you want to recruit specifically without getting any random curious families stopping by. @@ -54,13 +54,13 @@ We recommend **starting** studies as non-discoverable, so that you can pilot wit ============================= Share preview ============================= -Do you want other researchers to be able to preview your study? Check this box to make it possible for any logged-in Lookit researcher to try out your study. If you check the box, you will be able to share your preview link - e.g. on the Slack channel - to ask for feedback on your study from other researchers. This is generally a good idea as we could all use another pair of eyes to check on directions, stimuli, debriefing text, etc. Getting peer feedback ahead of time will generally substantially speed up the Lookit review process too. You can leave this unchecked if you're very concerned about being scooped. (My personal feeling is that no one has the time or energy to scoop you. See also: every line of our code is publicly available and has been for years...) +Do you want other researchers to be able to preview your study? Check this box to make it possible for any logged-in CHS researcher to try out your study. If you check the box, you will be able to share your preview link - e.g. on the Slack channel - to ask for feedback on your study from other researchers. This is generally a good idea as we could all use another pair of eyes to check on directions, stimuli, debriefing text, etc. Getting peer feedback ahead of time will generally substantially speed up the CHS review process too. You can leave this unchecked if you're very concerned about being scooped. (My personal feeling is that no one has the time or energy to scoop you. See also: every line of our code is publicly available and has been for years...) ============================= Study image ============================= -Thumbnail image that will be displayed to participants on Lookit's studies page. File must be an image-type and the dimensions must be square. Please keep the file size reasonable (<1 MB). If you submit too large an image file you may see an error "413 Request Entity Too Large." +Thumbnail image that will be displayed to participants on CHS's studies page. File must be an image-type and the dimensions must be square. Please keep the file size reasonable (<1 MB). If you submit too large an image file you may see an error "413 Request Entity Too Large." Sometimes your stimuli are a good basis for creating this image, or it can be something that conceptually represents your study or shows what it looks like to participate. @@ -91,7 +91,7 @@ Compensation Provide a description of any compensation for participation, including when and how participants will receive it and any limitations or eligibility criteria (e.g., only one gift card per participant, being in age range for study, child being visible in consent -video). Please see the `Terms of Use `_ for details +video). Please see the `Terms of Use `_ for details on allowable compensation and restrictions. If this field is left blank (which is okay if you're not providing compensation beyond the joy of participation) it will not be displayed to participants. @@ -197,11 +197,11 @@ If you select more than one study in the 'Must have participated' field, the chi Researchers may have different preferences and requirements about what counts as having 'participated' in a study; some researchers want to exclude children who have had any exposure to the study's stimuli, while others may only care about excluding children who have completed the whole session. Since the definition for these criteria has to be used for every study on the site, we have to use a definition of "participated" that operates in the same way for all studies. - We have chosen to treat participation in the strictest sense: **once a child begins a study by clicking the 'Participate Now' button, they are treated as having 'participated' in that study**, regardless of how far they get through the study session. For internal studies, we also check that the participant at least reaches the first frame (as opposed to clicking to start and then closing the page before it loads). + We have chosen to treat participation in the strictest sense: **once a child begins a study by clicking the 'Participate Now' button, they are treated as having 'participated' in that study**, regardless of how far they get through the study session. For internal studies, we also check that the participant at least reaches the first frame/trial (as opposed to clicking to start and then closing the page before it loads). If you want children to become ineligible for your study after doing it once, you can add the study itself to the 'must not have participated' list. However, keep in mind that as soon as the family starts a study session, the child will be treated as having 'particpated' (even if they don't finish; see :ref:`What counts as 'participated'? `). -If you are running an internal (Lookit) study then you have another option for warning families about participating in your study multiple times: you can implement a custom check at the beginning of the study session. Internal studies have access to all previous responses for that child and study through the :ref:`protocol generator function `. Using this function, you can check whether the child has already 'participated' (using your own definition, e.g. completed all frames, or reached a particular point in the study), and use that information to conditionally warn families about ineligibility (see this example in the :ref:`experiment runner protocol generator documentation `). If you implement your own check using the protocol generator, then you should not add the study itself to the 'must not have participated' criteria. +If you are running an internal Lookit study then you have another option for warning families about participating in your study multiple times: you can implement a custom check at the beginning of the study session. Internal Lookit studies have access to all previous responses for that child and study through the :ref:`Lookit protocol generator function `. Using this function, you can check whether the child has already 'participated' (using your own definition, e.g. completed all frames, or reached a particular point in the study), and use that information to conditionally warn families about ineligibility (see this example in the :ref:`experiment runner protocol generator documentation `). If you implement your own check using the protocol generator, then you should not add the study itself to the 'must not have participated' criteria. As with all of the eligibility criteria, if the child is not eligible due to the study's prior participation requirements, they can still choose to participate in the study, but they will see a warning below the 'Participate Now' button when the ineligible child is selected (see image below). The purpose of this warning is to make sure that families know that they may not be compensated and their data may not be used. @@ -227,7 +227,7 @@ If the child is not eligible based on the study's criteria expression, they will You may want to use the criteria expression to specify additional eligibility criteria beyond an age range - for instance, if your study is for a special population like kids with ASD or bilingual kids. In general, do **not** specify your age range here; participant eligibility checks will require the child meet the `minimum and maximum age cutoffs`_ AND these critera. -Every child in the Lookit database has a number of fields associated with it, ranging from gestational age to languages spoken in the home, which can be used in determining eligibility. In the study edit and create views, you can formulate your criteria expression as a boolean expression with embedded relational expressions, using a domain specific query language. +Every child in the CHS database has a number of fields associated with it, ranging from gestational age to languages spoken in the home, which can be used in determining eligibility. In the study edit and create views, you can formulate your criteria expression as a boolean expression with embedded relational expressions, using a domain specific query language. You can put together your expressions using the query fields below; the operators `AND`, `OR`, `NOT`, `<`, `<=`, `=`, `>`, and `>=`; and parentheses. If your expression is invalid you will see an error when you try to save your study. @@ -413,12 +413,12 @@ Language codes ========================================================= -Study protocol configuration (Internal studies) +Study protocol configuration (Lookit studies) ========================================================= This needs to be a valid JSON block describing the different frames (pages) of your study, and the sequence. This can be left blank at the time you initially create your study. For detailed information about specifying your study protocol, see `Building an Experiment`_. ========================================================= -Experiment runner code URL and version (Internal studies) +Experiment runner code URL and version (Lookit studies) ========================================================= The "Experiment runner code URL" is a link to the application you're using to enable participants to take a study. Right now, we just have one option, the `Ember Frame Player `_. It's an ember app that can talk to our API. All the frames in the experiment are defined in Ember and there is an exp-player component that can cycle through these frames. For details, see `Editing study type`_. You should leave this link as is, unless you have forked the Ember Frame Player Github repository (for instance, to create custom frame types) and want to use your fork instead. @@ -431,11 +431,11 @@ Study URL (External studies) ============================= The link that families should be redirected to when they click the "Participate now" button on a study detail page. For unscheduled/unmoderated studies, this will be the study itself (e.g. a Qualtrics survey). For moderated studies, it should be a link to a scheduling system (e.g. Calendly). -When the family clicks the "Participate now" button for external studies, the link will automatically include two pieces of information as URL query parameters: the hashed child ID ('child') and the response ID ('response'). This will allow you to automatically capture and record this information on the study/scheduling page, so that you can link the study responses and child's Lookit account without having to ask the family to enter addintional information. For example, if your Study URL is "\https://example.com", then the family will be directed to a link that has this format: +When the family clicks the "Participate now" button for external studies, the link will automatically include two pieces of information as URL query parameters: the hashed child ID ('child') and the response ID ('response'). This will allow you to automatically capture and record this information on the study/scheduling page, so that you can link the study responses and child's CHS account without having to ask the family to enter addintional information. For example, if your Study URL is "\https://example.com", then the family will be directed to a link that has this format: \https://example.com/?child=SG7JLN&response=d5c8f502-6588-46c8-84fa-a9657a44fe47 -It is up to the researcher to capture and record the URL query parameter values on the external website. Many online experiment/survey tools have documentation on how to do this (e.g. `Qualtrics `_). You can include your own URL query parameters in your Study URL and they will be retained along with the Lookit parameters. +It is up to the researcher to capture and record the URL query parameter values on the external website. Many online experiment/survey tools have documentation on how to do this (e.g. `Qualtrics `_). You can include your own URL query parameters in your Study URL and they will be retained along with the CHS parameters. =========================================== Scheduling Platform (External studies) diff --git a/docs/source/researchers-spam-prevention.rst b/docs/source/researchers-spam-prevention.rst index 3a30c42d..ea9ae6b6 100644 --- a/docs/source/researchers-spam-prevention.rst +++ b/docs/source/researchers-spam-prevention.rst @@ -94,21 +94,21 @@ successfully on CHS, so this is certainly an option to consider if your institut has an absolute ban on collecting visual or other corroborating evidence about your remote participants. -Option 2: The standard Lookit video consent process +Option 2: The standard video consent process ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The most straightforward way to meet this requirement, which the majority of studies -are currently using, is the Lookit video consent form, which captures +are currently using, is the video consent form, which captures webcam video of the parent/guardian stating that they agree to participate in the -study. In your compensation description, you can state that you require the child to +study. This option is available in Lookit and jsPsych studies. In your compensation description, you can state that you require the child to be visible in the consent video, or you can use other video data that you collect during the session. If you are running an external, asynchronous study (Bring-your-own Study Link), the easiest way to meet -this requirement is to prepare a 'hybrid' study that uses the Lookit experiment -builder to present the consent form, and then redirects to your external study. -`Here is an example study template `__ +this requirement is to prepare a 'hybrid' study that uses the either the Lookit or jsPsych experiment +builder to present the video consent form, and then redirects to your external study. +`Here is an example study template `__ that you can copy and use. There is also the option to record video of the entire session by using `the iframe method `__ to display your experiment in the same tab, without leaving the Children Helping Science @@ -223,7 +223,7 @@ procedures you use has to implement these suggestions. At a minimum, these proce Whether you are conducting an internal or external study, the CHS website saves information that you can use to review participants and check for patterns that may indicate spam. -**Consent manager:** Internal experiments and hybrid experiments that use the Lookit video consent frame +**Consent manager:** Internal experiments and hybrid experiments that use a video consent frame/trial can use the Consent Manager to screen for potential signs of bad-faith participation. .. image:: _static/img/spam_prevention_consent.png @@ -277,7 +277,7 @@ Use the following process to monitor and report potentially fraudulent participa us know when you submit a report. #. *DO NOT compensate the participant* until an admin has had a chance to review your case(s). Once you've heard back from an admin, you will know whether that participant has - been blocked from Lookit, whether they have concluded that the participant should + been blocked from CHS, whether they have concluded that the participant should be compensated, or whether there is another outcome needed. #. In your lab/research group, keep a running list of potentially problematic red flags you find in your lab - when it's an individual or small group causing diff --git a/docs/source/researchers-study-permissions.rst b/docs/source/researchers-study-permissions.rst index e1d200ab..8f145717 100644 --- a/docs/source/researchers-study-permissions.rst +++ b/docs/source/researchers-study-permissions.rst @@ -74,7 +74,7 @@ Admin Adding researchers to your study --------------------------------- -You can only give permissions to people in the Lab associated with this study. (If you're creating a study in the Sandbox Lab, you will be able to share with anyone on Lookit.) +You can only give permissions to people in the Lab associated with this study. (If you're creating a study in the Sandbox Lab, you will be able to share with anyone on CHS.) Halfway down the study detail page, you can see the researchers that have been added to your study. On the left, you can see researchers in this study's lab and search for a specific researcher. @@ -115,7 +115,7 @@ WRITE_STUDY_DETAILS Can make changes to the study using the edit study form - e.g., changing the protocol or age range. Note that if the study is already approved, making changes would automatically reject it; in this case, someone with WRITE_STUDY_DETAILS but not CHANGE_STUDY_STATUS would not be able to make changes. CHANGE_STUDY_STATUS - Can change the status of the study - starting or stopping data collection, submitting/retracting it for Lookit review, etc. + Can change the status of the study - starting or stopping data collection, submitting/retracting it for CHS review, etc. MANAGE_STUDY_RESEARCHERS Can add and remove other researchers in this lab to the various study groups, editing their permissions. @@ -138,7 +138,7 @@ CODE_STUDY_PREVIEW_CONSENT Can submit rulings about whether consent videos from preview data demonstrate valid informed consent. (They don’t really, but you might mock up the entire workflow from start to finish with preview data while planning your analyses and testing out your study.) CONTACT_STUDY_PARTICIPANTS - Can send email to participants via the Lookit platform. (This still does not grant direct access to participant email addresses.) + Can send email to participants via the CHS platform. (This still does not grant direct access to participant email addresses.) EDIT_STUDY_FEEDBACK Can create and edit feedback associated with participant responses, which is displayed to participants on the study history page. diff --git a/docs/source/researchers-style-guide.rst b/docs/source/researchers-style-guide.rst index aa5a26b7..18dc8e90 100644 --- a/docs/source/researchers-style-guide.rst +++ b/docs/source/researchers-style-guide.rst @@ -4,7 +4,7 @@ Style guide ################################## -We try to conform to some common stylistic, writing, and study design guidelines to provide a familiar and friendly experience across Lookit studies. +We try to conform to some common stylistic, writing, and study design guidelines to provide a familiar and friendly experience across CHS studies. ================================================ General language usage @@ -36,7 +36,7 @@ When choosing music or nursery rhymes for stimuli or background music, please co Eligibility criteria ========================== -In general, keep eligibility criteria as broad as possible for a positive family experience. The "hard" eligibility criteria you list on Lookit will include your age range and any characteristics that are critical to the study design or that are necessary for the study to make any sense to the child/family. For instance, if you have a question specifically about bilingual vs. trilingual kids, go ahead and require n_languages >= 2. But if you just routinely exclude premature babies or kids with trisomies from analysis, you can do that after the fact if needed since the study experience won't be affected. +In general, keep eligibility criteria as broad as possible for a positive family experience. The "hard" eligibility criteria you list on CHS will include your age range and any characteristics that are critical to the study design or that are necessary for the study to make any sense to the child/family. For instance, if you have a question specifically about bilingual vs. trilingual kids, go ahead and require n_languages >= 2. But if you just routinely exclude premature babies or kids with trisomies from analysis, you can do that after the fact if needed since the study experience won't be affected. Your eligibility criteria description should "translate" the minimum / maximum ages and any eligibility criteria expression into regular language, starting with "For..." - for example, "for 3-year-olds" or "for 8- through 12-month-olds who are exposed to least two languages at home." You can also include hard criteria that are not possible to check for programmatically, for instance if your IRB requires that participants live in a certain country. diff --git a/docs/source/researchers-update-code.rst b/docs/source/researchers-update-code.rst index 179c0821..d0823506 100644 --- a/docs/source/researchers-update-code.rst +++ b/docs/source/researchers-update-code.rst @@ -1,7 +1,7 @@ .. _updating-frameplayer-code: ############################################# -Updating the experiment runner +Updating the Lookit experiment runner ############################################# In the future, there may be changes in the Lookit experiment runner that you want your study to use - for instance, a bug fix for an issue your participants are encountering or a new frame you want to use. (By default, your study keeps chugging along using exactly the same code, so that updates can't change how your study works without your knowledge.) @@ -69,7 +69,7 @@ Here's how to update the code used: :alt: Example sha value filled in -4. Click "Save Changes". You will see a warning pop up if your study has already been approved, telling you it will be rejected automatically and require re-review. This is so that Lookit staff can review any new code you're using (in particular if you're using your own repo). +4. Click "Save Changes". You will see a warning pop up if your study has already been approved, telling you it will be rejected automatically and require re-review. This is so that CHS staff can review any new code you're using (in particular if you're using your own repo). .. image:: _static/img/update_code/click_save.png :alt: Save button diff --git a/docs/source/researchers-use-api.rst b/docs/source/researchers-use-api.rst index da740f7b..aa1bfebc 100644 --- a/docs/source/researchers-use-api.rst +++ b/docs/source/researchers-use-api.rst @@ -7,7 +7,7 @@ Using the API ======================= What is the API for? ======================= -Using the Lookit API allows you to programatically retrieve or update data (other than video data), rather than manually downloading JSON or CSV files from the Experimenter site. +Using the Children Helping Science API allows you to programatically retrieve or update data (other than video data), rather than manually downloading JSON or CSV files from the Experimenter site. Researchers do not need to use the API, but it is available if preferred to downloading data via the experimenter app. @@ -109,7 +109,7 @@ Authentication --------------- We are using a token-based HTTP Authentication scheme. -- Ask Lookit staff for a token. +- Ask CHS staff for a token. - Tokens are currently created via the Admin app accessible to staff only. Go to ``/__CTRL__/authtoken/token/add/``: @@ -131,13 +131,13 @@ We are using a token-based HTTP Authentication scheme. .. code-block:: bash - curl -X GET https://lookit.mit.edu/api/v1/users/ -H 'Authorization: Token 123456789abcdefghijklmnopqrstuvwxyz' + curl -X GET https://childrenhelpingscience.com/api/v1/users/ -H 'Authorization: Token 123456789abcdefghijklmnopqrstuvwxyz' - Here is an example of a POST request using curl, note the presence of the content-type header as well as the authorization header: .. code-block:: bash - curl -X POST http://lookit.mit.edu/api/v1/feedback/ -H "Content-Type: application/vnd.api+json" -H 'Authorization: Token abcdefghijklmnopqrstuvwxyzyour-token-here' -d '{"data": {"attributes": {"comment": "Test comment"}, "relationships": {"response": {"data": {"type": "responses","id": "91c15b81-bb25-437a-8299-13cf4c83fed6"}}},"type": "feedback"}}' + curl -X POST http://childrenhelpingscience.com/api/v1/feedback/ -H "Content-Type: application/vnd.api+json" -H 'Authorization: Token abcdefghijklmnopqrstuvwxyzyour-token-here' -d '{"data": {"attributes": {"comment": "Test comment"}, "relationships": {"response": {"data": {"type": "responses","id": "91c15b81-bb25-437a-8299-13cf4c83fed6"}}},"type": "feedback"}}' ------------ Pagination @@ -177,11 +177,11 @@ Permissions: Must be authenticated. You can only view children that have respon Ordering: Children can be sorted by birthday using the *ordering* query parameter. For example, to sort oldest to youngest: -GET http://lookit.mit.edu/api/v1/children/?ordering=birthday +GET http://childrenhelpingscience.com/api/v1/children/?ordering=birthday Add a '-' before birthday to sort youngest to oldest: -GET http://lookit.mit.edu/api/v1/children/?ordering=-birthday +GET http://childrenhelpingscience.com/api/v1/children/?ordering=-birthday *Sample Response:* @@ -595,8 +595,8 @@ Permissions: Must be authenticated. { "links": { - "first": "https://lookit-staging.mit.edu/api/v1/labs/?page=1", - "last": "https://lookit-staging.mit.edu/api/v1/labs/?page=1", + "first": "https://babieshelpingscience.com/api/v1/labs/?page=1", + "last": "https://babieshelpingscience.com/api/v1/labs/?page=1", "next": null, "prev": null, "meta": { @@ -611,15 +611,15 @@ Permissions: Must be authenticated. "id": "a2a7383c-cb58-4d78-ac00-23283a762dec", "attributes": { "name": "Demo lab", - "institution": "Lookit", + "institution": "Children Helping Science", "principal_investigator_name": "Sample Name", - "lab_website": "https://lookit.mit.edu/", - "description": "This is a sample lab researchers are added to upon joining Lookit. It contains several demo\r\n studies you will be able to see.", + "lab_website": "https://childrenhelpingscience.com/", + "description": "This is a sample lab researchers are added to upon joining CHS. It contains several demo\r\n studies you will be able to see.", "approved_to_test": true, "pk": 2 }, "links": { - "self": "https://lookit-staging.mit.edu/api/v1/labs/a2a7383c-cb58-4d78-ac00-23283a762dec/" + "self": "https://babieshelpingscience.com/api/v1/labs/a2a7383c-cb58-4d78-ac00-23283a762dec/" } }, { @@ -635,7 +635,7 @@ Permissions: Must be authenticated. "pk": 3 }, "links": { - "self": "https://lookit-staging.mit.edu/api/v1/labs/1c54bccd-5f3d-4dd1-967a-0f7c0565d76d/" + "self": "https://babieshelpingscience.com/api/v1/labs/1c54bccd-5f3d-4dd1-967a-0f7c0565d76d/" } } ] @@ -657,15 +657,15 @@ Permissions: Must be authenticated. "id": "a2a7383c-cb58-4d78-ac00-23283a762dec", "attributes": { "name": "Demo lab", - "institution": "Lookit", + "institution": "Children Helping Science", "principal_investigator_name": "Sample Name", - "lab_website": "https://lookit.mit.edu/", - "description": "This is a sample lab researchers are added to upon joining Lookit. It contains several demo\r\n studies you will be able to see.", + "lab_website": "https://childrenhelpingscience.com/", + "description": "This is a sample lab researchers are added to upon joining CHS. It contains several demo\r\n studies you will be able to see.", "approved_to_test": true, "pk": 2 }, "links": { - "self": "https://lookit-staging.mit.edu/api/v1/labs/a2a7383c-cb58-4d78-ac00-23283a762dec/" + "self": "https://babieshelpingscience.com/api/v1/labs/a2a7383c-cb58-4d78-ac00-23283a762dec/" } } } diff --git a/docs/source/tutorial-access.rst b/docs/source/tutorial-access.rst index bf51ab1b..dc2afd4a 100644 --- a/docs/source/tutorial-access.rst +++ b/docs/source/tutorial-access.rst @@ -1,40 +1,40 @@ .. _tutorial: -###################################### -1. Accessing Lookit and community -###################################### +#################################################### +1. Accessing Children Helping Science and community +#################################################### Welcome ---------- -Welcome to the Lookit tutorial! In this tutorial you will work through a series of exercises intended to build all the skills you'll need to run your own studies on Lookit. By the end of the tutorial, you will be able to create a participant-friendly study on Lookit that includes a survey and test trials, assign children to different experimental conditions, test and troubleshoot your study, download study data, contact participants, use and even edit this documentation! +Welcome to the Children Helping Scienc (CHS) tutorial! In this tutorial you will work through a series of exercises intended to build all the skills you'll need to run your own Lookit study on the CHS platform. By the end of the tutorial, you will be able to create a participant-friendly study that includes a survey and test trials, assign children to different experimental conditions, test and troubleshoot your study, download study data, contact participants, use and even edit this documentation! -This tutorial was written for researchers who will be building internal Lookit studies, but most steps are also applicable for researchers who will be sharing a link to an external study instead. +This tutorial was written for researchers who will be building a Lookit study, but most steps are also applicable for researchers who will be building a jsPsych study or sharing a link to an external study instead. -This tutorial should take between 5 and 10 hours to complete, and does not assume any familiarity with programming. There is a mix of step-by-step directions and exercises to complete. It's the best way to get started using Lookit, and a good first task to give to an RA who will be designing studies. Note that it does **not** cover issues like participant recruitment and IRB approval, just the technical side of things. +This tutorial should take between 5 and 10 hours to complete, and does not assume any familiarity with programming. There is a mix of step-by-step directions and exercises to complete. It's the best way to get started with creating and managing a Lookit study, and a good first task to give to an RA who will be designing studies. Note that it does **not** cover issues like participant recruitment and IRB approval, just the technical side of things. Throughout the tutorial, you may want to have the tutorial open in one browser window while you complete the tasks in another window. -To get started, you need to get on Lookit! +To get started, you need to get on Children Helping Science! -Step 1: Join the Lookit Slack workspace ----------------------------------------- -Fill out `this form `_ to join the Lookit Slack workspace. If you haven't used Slack before, you'll need to create an account; otherwise it will be added to your other workspaces. +Step 1: Join the Lookit/CHS Slack workspace +--------------------------------------------- +Fill out `this form `_ to join the Lookit/CHS Slack workspace. If you haven't used Slack before, you'll need to create an account; otherwise it will be added to your other workspaces. -Slack is a messaging platform that serves as the primary tool for communication about development and use of the Lookit platform. The Lookit Slack workspace is essentially a collection of "chat rooms" for various topics. You can use Slack via a web interface, a desktop app, and/or on your phone. +Slack is a messaging platform that serves as the primary tool for communication about development and use of the CHS platform. The Lookit/CHS Slack workspace is essentially a collection of "chat rooms" for various topics. You can use Slack via a web interface, a desktop app, and/or on your phone. Step 2 (optional): Sign up for the lookit-research mailing list ---------------------------------------------------------------- -We periodically (about once a month) send out updates about Lookit. If you'd prefer to get +We periodically (about once a month) send out updates about Lookit/CHS. If you'd prefer to get these updates via email instead of only on Slack, please `sign up for this list `__. -Step 3: Create your Lookit account +Step 3: Create your CHS account ---------------------------------------- -1. Go to `the researcher registration form `__. Fill out the form and click "create account." +1. Go to `the researcher registration form `__. Fill out the form and click "create account." -2. You'll be taken to a page like this to set up two-factor authentication (2FA), which you'll need in order to access the researcher section of Lookit. If you haven't already, download the Google Authenticator app on your phone. Then follow the directions on this page to activate 2FA for your account. +2. You'll be taken to a page like this to set up two-factor authentication (2FA), which you'll need in order to access the researcher section of CHS. If you haven't already, download the Google Authenticator app on your phone. Then follow the directions on this page to activate 2FA for your account. .. image:: _static/img/login/login_2fa_setup.png :alt: Two-factor authentication setup screen @@ -50,7 +50,7 @@ Step 3: Create your Lookit account Where to go for help --------------------- -* If you need help as you're completing this tutorial - or later when you're creating and running studies on Lookit - the best place to go is the Lookit Slack workspace. Check the 'researchers' or 'tech support' channels to see if someone has asked the same question before! If not, post in one of these channels to get support from Lookit staff and/or other researchers who may have dealt with the same issue. (If you need help specifically from Lookit staff, under almost all circumstances the question should *still* go in one of the public channels rather than a private chat, so that other people can see the answers.) +* If you need help as you're completing this tutorial - or later when you're creating and running studies on CHS - the best place to go is the Lookit/CHS Slack workspace. Check the 'researchers' or 'tech support' channels to see if someone has asked the same question before! If not, post in one of these channels to get support from CHS staff and/or other researchers who may have dealt with the same issue. (If you need help specifically from CHS staff, under almost all circumstances the question should *still* go in one of the public channels rather than a private chat, so that other people can see the answers.) Even if you're not totally stumped, it's worth checking in on Slack to get ideas and resources! For example, you might ask if anyone has an R script for analyzing their data that you could use as a starting point, or run a draft debriefing by other researchers to get advice on wording. @@ -59,10 +59,10 @@ Where to go for help Exercises ---------- -1. Say hello in the "tutorial" channel of the Lookit Slack workspace. +1. Say hello in the "tutorial" channel of the Lookit/CHS Slack workspace. -2. Get a feel for the participant experience by trying out one of the "template" studies! At https://lookit.mit.edu/exp/studies/, find a study labeled TEMPLATE from the lab "Demo lab." Click on it and then click "Preview study" to see how it would work if you were a +2. Get a feel for the participant experience by trying out one of the "template" studies! At https://childrenhelpingscience.com/exp/studies/, find a study labeled TEMPLATE from the lab "Demo lab." Click on it and then click "Preview study" to see how it would work if you were a participant! You may need to add a child profile and complete your demographic survey, -which you can do from https://lookit.mit.edu/account/manage/. +which you can do from https://childrenhelpingscience.com/account/manage/. -3. [Extra credit] If you have young kids, take part in a currently-running study on the production server (https://lookit.mit.edu) - the experience is different with a squirming kid in your lap! Please enter only actual, not made-up, data on the production server - you are participating in real research here. (If you're curious but don't have a child in the age range, feel free to contact the lab and ask if it's ok to check it out--generally they can deal with that easily, but a heads up is helpful.) +3. [Extra credit] If you have young kids, take part in a currently-running study on the production server (https://childrenhelpingscience.com) - the experience is different with a squirming kid in your lap! Please enter only actual, not made-up, data on the production server - you are participating in real research here. (If you're curious but don't have a child in the age range, feel free to contact the lab and ask if it's ok to check it out--generally they can deal with that easily, but a heads up is helpful.) diff --git a/docs/source/tutorial-contributing.rst b/docs/source/tutorial-contributing.rst index 3beab508..5cfed5f6 100644 --- a/docs/source/tutorial-contributing.rst +++ b/docs/source/tutorial-contributing.rst @@ -2,17 +2,17 @@ 2. Contributing to the codebase ################################## -Lookit is an open-source project, meaning that the code is freely available and anyone can contribute. This includes the actual Lookit platform, the components used in studies, AND the documentation you're reading now. Community contributions and input are critical to the project! +Children Helping Science (CHS) is an open-source project, meaning that the code is freely available and anyone can contribute. This includes the actual CHS platform, the components used in Lookit and jsPsych studies, AND the documentation you're reading now. Community contributions and input are critical to the project! In this section, you'll make your first "Pull request" (PR) to propose a change to the documentation [#]_. Your change will just be to add yourself to a list of tutorial participants. This way, you'll be comfortable making substantive changes whenever you're ready! -Later on in the tutorial, if you come across something confusing or you have an idea to improve the instructions or exercises, you can follow these same instructions to fix it. Although we recognize that most Lookit users will not be ready to contribute to the code that makes Lookit run, editing the documentation is something everyone can do! We need your help to keep it up-to-date and constantly improving! +Later on in the tutorial, if you come across something confusing or you have an idea to improve the instructions or exercises, you can follow these same instructions to fix it. Although we recognize that most CHS users will not be ready to contribute to the code that makes CHS run, editing the documentation is something everyone can do! We need your help to keep it up-to-date and constantly improving! .. _First PR: Making your first pull request (PR) ------------------------------------ -All of the code that makes Lookit run is stored on GitHub, which makes it easy to keep track of changes over time and merge changes from many collaborators. There are several code repositories or "repos" that house different pieces of the project. Here we are going to walk through making a change to the documentation repo, `lookit-docs `_. Making a "pull request" or PR is a way to ask that your edits be incorporated into the main codebase. It's actually not too hard, and it's a great way to contribute! +All of the code that makes CHS run is stored on GitHub, which makes it easy to keep track of changes over time and merge changes from many collaborators. There are several code repositories or "repos" that house different pieces of the project. Here we are going to walk through making a change to the documentation repo, `lookit-docs `_. Making a "pull request" or PR is a way to ask that your edits be incorporated into the main codebase. It's actually not too hard, and it's a great way to contribute! Don't worry about making a mistake and somehow messing up the documentation - you can't directly edit the "official" version of the files! There is always a review process before your changes are merged in. @@ -56,7 +56,7 @@ Don't worry about making a mistake and somehow messing up the documentation - yo .. image:: _static/img/tutorial/pr_view.png :alt: PR view on GitHub -The PR will now be in a list for Lookit staff to review. You can return to this page to check on it; if you have allowed Github to send notifications via email (the default), you will also get an email about any activity. You can also see your PR in the list by going to https://github.com/lookit/lookit-docs and clicking on "Pull requests": +The PR will now be in a list for CHS staff to review. You can return to this page to check on it; if you have allowed Github to send notifications via email (the default), you will also get an email about any activity. You can also see your PR in the list by going to https://github.com/lookit/lookit-docs and clicking on "Pull requests": .. image:: _static/img/tutorial/pr_tab.png :alt: See all PRs on GitHub @@ -344,25 +344,27 @@ List of tutorial participants Checking for and creating issues on Github ------------------------------------------- -What if you notice a problem while using Lookit, or something unclear in the documentation, but it's not something you know how to fix? Or what if you find yourself wishing there were a particular feature that would make your research easier? +What if you notice a problem while using CHS, or something unclear in the documentation, but it's not something you know how to fix? Or what if you find yourself wishing there were a particular feature that would make your research easier? .. image:: _static/img/tutorial/issues_tab.png :alt: See all issues on GitHub To track bug reports and feature requests, we use GitHub **issues**. You can see issues by clicking on the "Issues" tab in the appropriate repository or "repo": -- `lookit-api `_ is the repo for the Lookit site: issues with anything to do with participant login or data, how current and past studies are displayed to participants, how you view data and manage your studies +- `lookit-api `_ is the repo for the CHS site: issues with anything to do with participant login or data, how current and past studies are displayed to participants, how you view data and manage your studies. -- `ember-lookit-frameplayer `_ is the repo for the experiment components themselves: issues with how particular frames behave, frames you'd find useful, counterbalancing/condition assignment, etc. +- `ember-lookit-frameplayer `_ is the repo for the Lookit experiment components: issues with how particular Lookit frames behave, frames you'd find useful, counterbalancing/condition assignment, etc. + +- `lookit-jspsych `_ is the repo for the CHS-specific parts of jsPsych experiments: issues with how jsPsych studies run *on CHS*, and CHS-specific features such as the consent and exit surveys and video recording features. - `lookit-docs `_ is the repo for the documentation: anything about the docs you're reading now! -To request a feature or report a bug, first search the existing issues to see if your idea is already there. +To request a feature or report a bug, first search the existing issues to see if your problem or idea is already there. .. image:: _static/img/tutorial/search_issues.png :alt: Search issues on GitHub -If so, comment on it or add a thumbs-up reaction so Lookit staff know there's more interest! If not, click the green "New issue" button at the top right. +If so, comment on it or add a thumbs-up reaction so CHS staff know there's more interest! If not, click the green "New issue" button at the top right. .. image:: _static/img/tutorial/new_issue.png :alt: Create new issue on GitHub @@ -382,7 +384,7 @@ Your issue will now have a number assigned to it and will be listed in the issue .. image:: _static/img/tutorial/issue.png :alt: Issue on GitHub -Lookit staff may respond to ask for further information, schedule it for future development, and/or wait for community feedback about the idea to gauge demand. +CHS staff may respond to ask for further information, schedule it for future development, and/or wait for community feedback about the idea to gauge demand. Exercises ---------- diff --git a/docs/source/tutorial-first-study.rst b/docs/source/tutorial-first-study.rst index 3eb14f5a..05148be8 100644 --- a/docs/source/tutorial-first-study.rst +++ b/docs/source/tutorial-first-study.rst @@ -1,15 +1,15 @@ -################################## -3. Setting up your first study -################################## +###################################### +3. Setting up your first Lookit study +###################################### -In this section, you will be creating your first study on Lookit. You will learn how to find and use experiment components, specify your protocol, and test out and troubleshoot your study. +In this section, you will be creating your first Lookit study. You will learn how to find and use experiment components, specify your protocol, and test out and troubleshoot your study. -This section is designed for researchers planning to build an internal Lookit study. If your first study involves a link to an experiment hosted somewhere else, skip to part 6 of this tutorial. +This section is designed for researchers planning to build an internal Lookit study. If your first study will be written with jsPsych, then you can see the :ref:`jspsych tutorial `. Or if your study involves a link to an experiment hosted somewhere else, skip to part 6 of this tutorial. Step 1: Clone the 'Lookit tutorial part 1' study ------------------------------------------------- -To get started, log in to Lookit as an experimenter. (Go to ``_ to log in.) +To get started, log in to CHS as an experimenter. (Go to ``_ to log in.) You should see a few studies you automatically have access to, including one called "Lookit tutorial part 1". Click on that to open up the study detail page: @@ -63,7 +63,7 @@ Step 2: Preview your study (and learn a bit about JSON on the way) .. _Building study dependencies: -You may have noticed that below your thumbnail and basic study info, there's a section about the "status" of your study. This section is where you will submit your study for approval by Lookit staff when it's ready, and start and stop data collection. +You may have noticed that below your thumbnail and basic study info, there's a section about the "status" of your study. This section is where you will submit your study for approval by CHS staff when it's ready, and start and stop data collection. This section also shows whether your "experiment runner" is "built" yet. You should see a bar like this: @@ -75,9 +75,9 @@ Click the "Build experiment runner" button. You should see a notification at the .. image:: _static/img/tutorial/scheduled_for_preview.png :alt: Scheduled for build notification -What is this "experiment runner"? When you create a study on Lookit, you specify what types of pages or "frames" to use, and provide parameters for each - for example you supply the text for an instructions page, videos or images to show in a preferential looking trial, audio and images for a storybook page, and so on. The Lookit frame player interprets this information and turns it into an interactive study families can participate in. There's code behind the scenes, which you don't have to deal with, to handle that interpretation and to make each page "go" (saying what each button should do and what data to collect, arranging and starting/stopping video, etc.) Rather than all studies sharing that code, each study gets its own siloed little environment called a Docker image where it will run. +What is this "experiment runner"? When you create a Lookit study, you specify what types of pages or "frames" to use, and provide parameters for each - for example you supply the text for an instructions page, videos or images to show in a preferential looking trial, audio and images for a storybook page, and so on. The Lookit frameplayer interprets this information and turns it into an interactive study families can participate in. There's code behind the scenes, which you don't have to deal with, to handle that interpretation and to make each page "go" (saying what each button should do and what data to collect, arranging and starting/stopping video, etc.) Rather than all studies sharing that code, each study gets its own siloed little environment called a Docker image where it will run. -When you click "Build experiment runner," you are creating that Docker image and installing all the necessary code on it - the Lookit frameplayer and the other libraries it depends on. This way, as we continue expanding the Lookit frameplayer code, your study will continue to run exactly as you initially designed and tested it, unless you choose to update what code your study uses and build dependencies again - for instance to take advantage of a new feature or a bug fix. You also have the advanced option of telling Lookit to use your own code instead of the standard Lookit code - for instance if your work needs a very specialized type of test trial that you want to write your own frame for. +When you click "Build experiment runner," you are creating that Docker image and installing all the necessary code on it - the Lookit frameplayer and the other libraries it depends on. This way, as we continue expanding the Lookit frameplayer code, your study will continue to run exactly as you initially designed and tested it, unless you choose to update what code your study uses and build dependencies again - for instance to take advantage of a new feature or a bug fix. You also have the advanced option of telling the CHS platform to use your own code instead of the standard Lookit experiment runner code - for instance if your work needs a very specialized type of test trial that you want to write your own frame for. It will probably take about 10 minutes to build the experiment runner (you can wait for the email or refresh the page to see if it's done yet). @@ -161,7 +161,7 @@ Here are several things that are almost, but not quite, valid JSON objects. Copy

There are several things to fix here - if you changed something and got a different error message, you're probably on the right track! Work down the list of requirements for JSON from the overview, and note that the "true" value is all lowercase.

-OK, congrats on learning all about JSON! Your study should be ready to preview by now. You should have an email in your inbox from Lookit about this, and if you refresh the page you're on, you should see something like this: +OK, congrats on learning all about JSON! Your study should be ready to preview by now. You should have an email in your inbox from CHS about this, and if you refresh the page you're on, you should see something like this: .. image:: _static/img/tutorial/preview_built.png :alt: Preview built status display @@ -171,11 +171,11 @@ Click on "Preview study" near the top of the page: .. image:: _static/img/tutorial/preview_button.png :alt: Preview button -This will take you to a "study detail" page just like the one participants see when they click on a study at lookit.mit.edu/studies. You will need to register at least one child and respond to the demographic survey (you don't need to respond to all questions, or use real information). Then you will be able to click "Preview now" to proceed through the study as a participant. It's a rough, abbreviated implementation of one condition from `Schulz, Bonawitz, and Griffiths (2007) `_ - you'll read through a storybook about Bunny, who sometimes gets a tummyache, and eventually answer a question about what makes her tummy hurt. +This will take you to a "study detail" page just like the one participants see when they click on a study at childrenhelpingscience.com/studies. You will need to register at least one child and respond to the demographic survey (you don't need to respond to all questions, or use real information). Then you will be able to click "Preview now" to proceed through the study as a participant. It's a rough, abbreviated implementation of one condition from `Schulz, Bonawitz, and Griffiths (2007) `_ - you'll read through a storybook about Bunny, who sometimes gets a tummyache, and eventually answer a question about what makes her tummy hurt. .. admonition:: Note - Video is collected during previews! Like other data, it's only accessible to people who have appropriate permissions. That does include a few Lookit staff in addition to researchers working on your study. We don't do anything with the video and are very unlikely to even see it, but it *is* in principle possible - so please wear clothes while testing, don't sit in front of your really cool poster of your social security number, etc. Or cover your webcam. + Video is collected during previews! Like other data, it's only accessible to people who have appropriate permissions. That does include a few CHS staff in addition to researchers working on your study. We don't do anything with the video and are very unlikely to even see it, but it *is* in principle possible - so please wear clothes while testing, don't sit in front of your really cool poster of your social security number, etc. Or cover your webcam. Step 3: Get comfortable making changes to how your study works @@ -419,7 +419,7 @@ You should see something like this: .. admonition:: Browser-dependent behavior - Webcam access functionality, external resource loading, or other features may work slightly differently across web browsers, especially as they are updated over time. It is always worth previewing your studies in both Firefox and Chrome, which are the browsers Lookit currently officially supports. For your own privacy, we strongly recommend not using Chrome more than you have to. (We also recommend flossing and having Easter egg hunts as a year-round activity, but these are getting further from our domain.) + Webcam access functionality, external resource loading, or other features may work slightly differently across web browsers, especially as they are updated over time. It is always worth previewing your studies in both Firefox and Chrome, which are the browsers CHS currently officially supports. For your own privacy, we strongly recommend not using Chrome more than you have to. (We also recommend flossing and having Easter egg hunts as a year-round activity, but these are getting further from our domain.) .. admonition:: Advanced developer tool features @@ -427,7 +427,7 @@ You should see something like this: Now that you've gotten your web console open, you'll see a bunch of information in it. This is generally of most interest if something is going wrong and you're not sure what. You can see events that are being logged as you proceed through the study as well as any errors. Some of these errors are ok to ignore - e.g. here are a few current ones due to known but harmless bugs: -* On the staging server there is a known bug that the fontawesome library doesn't load properly (but it does on "production" - the real Lookit site) - so you may see some errors that a resource failed to load properly, like this: +* On the staging server there is a known bug that the fontawesome library doesn't load properly (but it does on "production" - the real CHS site) - so you may see some errors that a resource failed to load properly, like this: .. image:: _static/img/tutorial/fa_error.png :alt: Fontawesome error example @@ -532,7 +532,7 @@ Each frame you define on Lookit has to have a property called ``kind`` which say * the ``storybook-causal`` frame has kind ``group`` (this is a special kind of frame, documented :ref:`here `) * the frames **within** the ``storybook-causal`` frame have kind ``exp-lookit-images-audio`` (this is added to each frame in the ``frameList`` as part of the ``commonFrameProperties``) -In addition to this tutorial and the information in this documentation about how to set up a study on Lookit, there is detailed information available about each of the "frames" you can use in your Lookit study. In the :ref:`experiment runner docs ` you can browse the options and learn about options for customizing each type of frame. +In addition to this tutorial and the information in this documentation about how to set up a Lookit study on CHS, there is detailed information available about each of the "frames" you can use in your Lookit study. In the :ref:`experiment runner docs ` you can browse the options and learn about options for customizing each type of frame. Let's take a look at the ``exp-lookit-images-audio`` documentation to see what options we have. Find it on the left sidebar and click on it. Here's what you'll see: diff --git a/docs/source/tutorial-manage-data.rst b/docs/source/tutorial-manage-data.rst index 57378eb7..ee74a80e 100644 --- a/docs/source/tutorial-manage-data.rst +++ b/docs/source/tutorial-manage-data.rst @@ -11,7 +11,7 @@ Managing access to your study: add a collaborator Rarely will you be working completely alone! Usually you will want multiple people to have access to any particular study: you may have a few people working together to get the protocol just right, as well as several RAs checking consent and sending feedback to participants. -We very strongly recommend this model rather than sharing credentials for a lab-wide account. This way, each individual can get permissions on just the studies they actually need access to - not everything your lab has ever done. When temporary staff like undergrad RAs move on, you can just remove them from the study, instead of distributing a new password to everyone remaining in the lab. Plus, there is logging built into Lookit that keeps track of things like who did what when - including who approved consent recordings - that may be useful to you. +We very strongly recommend this model rather than sharing credentials for a lab-wide account. This way, each individual can get permissions on just the studies they actually need access to - not everything your lab has ever done. When temporary staff like undergrad RAs move on, you can just remove them from the study, instead of distributing a new password to everyone remaining in the lab. Plus, there is logging built into CHS that keeps track of things like who did what when - including who approved consent recordings - that may be useful to you. Try it out! Add another person to your tutorial study. On the study page in the Experimenter interface, scroll down to "Manage Researchers": @@ -55,7 +55,7 @@ So far, we have tried out our studies via the "preview study" button on the stud - When you preview a study, there is an "is_preview" field of the data collected that's set to true - otherwise it's false. Data collected from previewing is marked when you view consent videos or individual responses, and this field is available in the all-response downloads. -- Only Lookit researchers with appropriate permissions can preview the study. (Either the researcher needs to have read permissions for the study, or the study needs to be set to have a shared preview - then any researcher can access it.) Anyone with a child registered on Lookit can participate in a study. +- Only CHS researchers with appropriate permissions can preview the study. (Either the researcher needs to have read permissions for the study, or the study needs to be set to have a shared preview - then any researcher can access it.) Anyone with a child registered on CHS can participate in a study. Other than that, the experience is exactly the same, by design - so that you know exactly how your study will work. You see the same messages about whether your child is eligible, customization based on the child or past responses works the same way, and you use the same experiment runner. @@ -66,23 +66,23 @@ If you just want other researchers to be able to preview your study to give feed But what about when you actually want to start data collection? -At that point, you will "submit" your study for approval by Lookit staff. We won't practice this piece, but so you know what to expect, you can look through the +At that point, you will "submit" your study for approval by CHS staff. We won't practice this piece, but so you know what to expect, you can look through the :ref:`information about submitting your study `. .. admonition:: Why the manual approval process? - From a participant's standpoint, Lookit is a unified platform, even though there are studies from a variety of research labs. This is great for participant recruitment! But it also means we're all sharing a reputation. Someone else's study that upsets or (without adequate precautions) deceives children, that baffles parents, or that just doesn't work will affect how interested families are in your study, too. Based on our early experience with researchers using Lookit, we strongly expect that a quick review will catch substantive issues often enough to be worth putting everyone through. If you are making changes to an existing study, review is either not required (if only changing certain fields like the age range/eligibility criteria) or is very quick. + From a participant's standpoint, Children Helping Science is a unified platform, even though there are studies from a variety of research labs. This is great for participant recruitment! But it also means we're all sharing a reputation. Someone else's study that upsets or (without adequate precautions) deceives children, that baffles parents, or that just doesn't work will affect how interested families are in your study, too. Based on our early experience with researchers using CHS, we strongly expect that a quick review will catch substantive issues often enough to be worth putting everyone through. If you are making changes to an existing study, review is either not required (if only changing certain fields like the age range/eligibility criteria) or is very quick. Create some data to play with (Internal studies) -------------------------------------------------------- -Because we don't want to clog up the production server with fake responses from researchers trying out Lookit, we'll do this section on the staging server, which is a separate sandbox environment that looks a lot like Lookit but doesn't have any real participant data. This is also where we try out new features before deploying them to production. +Because we don't want to clog up the production server with fake responses from researchers trying out CHS, we'll do this section on the staging server, which is a separate sandbox environment that looks a lot like CHS but doesn't have any real participant data. This is also where we try out new features before deploying them to production. Go ahead and create an experimenter account on the staging server following the :ref:`login directions `. If you did the first part of the tutorial, you'll already have a participant account on the staging server - use a different email address for your staging experimenter account. -First, let's actually participate in another study! Go to the **staging** server studies page, ``_, and select the study "Apples to oranges." This is a short study just to demo the data collection process. You can participate using your experimenter account; you may need to make a child profile and/or fill out a demographic survey before participating. Proceed all the way through this study! +First, let's actually participate in another study! Go to the **staging** server studies page, ``_, and select the study "Apples to oranges." This is a short study just to demo the data collection process. You can participate using your experimenter account; you may need to make a child profile and/or fill out a demographic survey before participating. Proceed all the way through this study! -Now switch back to the Experimenter interface. Note: you can toggle between Lookit (the participant-facing section) and Experimenter (the researcher-facing section) at any time via the top navbar if you are logged in as an experimenter: +Now switch back to the Experimenter interface. Note: you can toggle between the participant-facing and "Experimenter" (researcher-facing) sections of the Children Helping Science website at any time via the top navbar if you are logged in as an experimenter: .. image:: _static/img/tutorial/lookit_view.png :alt: Participant-facing Apples and Oranges detail page @@ -93,7 +93,7 @@ Now switch back to the Experimenter interface. Note: you can toggle between Look Get access to the "Apples to Oranges" study ------------------------------------------------- -You are able to **see** the "Apples to Oranges" study listed on the Experimenter site on lookit-staging.mit.edu because you automatically get read-only permissions for studies within the Demo lab. However, you can't automatically see any participant data! (This is on purpose - it's not possible to grant lab-wide permissions to actual data, you have to actively add people to individual studies.) +You are able to **see** the "Apples to Oranges" study listed on the Experimenter site on babieshelpingscience.com because you automatically get read-only permissions for studies within the Demo lab. However, you can't automatically see any participant data! (This is on purpose - it's not possible to grant lab-wide permissions to actual data, you have to actively add people to individual studies.) Post in the Slack #tutorial channel and we'll add you as a researcher so you can see everything! Then, at the top of the "Apples to oranges" page, click on "View responses": @@ -113,7 +113,7 @@ The first page you see when you click "View Responses" is called the Consent Man As data comes in, your first step will always be to check whether the parent provided informed consent. You do that here in the consent manager, which by default shows you the "pending" consent videos for review. In the left column, you'll click on each session to bring up the associated consent video at the center. You can use the dropdown menu to decide whether to "accept" (mark this as valid consent) or "reject" (mark as invalid consent) each video. If you want to add any notes about the consent video, you can record comments in the text box beneath the video - for instance, you might note that there was a technical problem with the video, but you contacted the parent to confirm consent. -For now, just mark your own video that you just made as "Accepted." Then click "Submit Rulings & Comments". This saves your consent coding to the Lookit server. (In case it matters to your IRB: A record of which logged-in user made each consent determination and when is also stored.) +For now, just mark your own video that you just made as "Accepted." Then click "Submit Rulings & Comments". This saves your consent coding to the CHS server. (In case it matters to your IRB: A record of which logged-in user made each consent determination and when is also stored.) In the Consent Manager, you can now use the top drop-down menu to view currently "accepted" responses, and you should be able to see your own video there: @@ -161,7 +161,7 @@ Under "All responses," you can download JSON or CSV files with data about all re .. image:: _static/img/tutorial/all_responses.png :alt: All responses view -Analyzing the data collected is, in general, outside the scope of this tutorial as it will vary substantially by lab/project - although we hope that you will share your scripts and processes for analyzing Lookit data to help other researchers! The exercises below can be solved by manual inspection of the CSV (or JSON) data, although you are also welcome to set up a script in your language of choice to get a head start on real data processing. +Analyzing the data collected is, in general, outside the scope of this tutorial as it will vary substantially by lab/project - although we hope that you will share your scripts and processes for analyzing CHS data to help other researchers! The exercises below can be solved by manual inspection of the CSV (or JSON) data, although you are also welcome to set up a script in your language of choice to get a head start on real data processing. Exercises ~~~~~~~~~~ diff --git a/docs/source/tutorial-next-steps.rst b/docs/source/tutorial-next-steps.rst index 9d41e626..acda5864 100644 --- a/docs/source/tutorial-next-steps.rst +++ b/docs/source/tutorial-next-steps.rst @@ -7,22 +7,23 @@ You're in the home stretch! Thanks for sticking with it. We'll wrap up by revisi 1. Making your first Github issue or comment --------------------------------------------- -Now that you've created your first two studies and practiced some of the day-to-day study management tasks you'll do on Lookit, you've almost certainly thought of something you wish you could do easily on Lookit - or just something you found confusing. +Now that you've created your first two studies and practiced some of the day-to-day study management tasks you'll do on CHS, you've almost certainly thought of something you wish you could do easily on CHS - or just something you found confusing. -As we discussed at the start of the tutorial, Lookit is an open-source project meant to serve the research community, and we need everyone to contribute to keep making it better! One of the best ways to contribute is to document the problems you encounter and the ideas you have for improvement. +As we discussed at the start of the tutorial, CHS is an open-source project meant to serve the research community, and we need everyone to contribute to keep making it better! One of the best ways to contribute is to document the problems you encounter and the ideas you have for improvement. Let's practice that now. First, think of either a problem you encountered or something you'd like to see, in any of these categories: - The `documentation `_, including the tutorial you're reading now -- The `Lookit platform `_, including login/account system and the experimenter inferface (study management, data viewing/download, consent coding, participant messaging, etc.) -- The `frameplayer `_, including what frames are available to you to use in designing your studies and how they function +- The `CHS platform `_, including login/account system and the experimenter inferface (study management, data viewing/download, consent coding, participant messaging, etc.) +- The `Lookit frameplayer `_, including what frames are available to you to use in designing your Lookit studies and how they function +- The `jsPsych experiment runner `_, including what plugins/extensions are available to you to use in designing your jsPsych studies and how they function :ref:`Review the section on Github issues` and click the appropriate link above to view the current issue list. Either create a new issue if your idea/problem isn't already described, or comment on an existing issue! 2. Some advanced options to be aware of --------------------------------------- -Although we hope this tutorial has provided a solid introduction to designing and managing your Lookit study, it doesn't cover everything it's possible to do! The goal is that upon completion, you are ready to learn about more advanced options using the rest of the documentation. Here are some examples of topics you may be interested in exploring as you continue: +Although we hope this tutorial has provided a solid introduction to designing and managing your CHS study, it doesn't cover everything it's possible to do! The goal is that upon completion, you are ready to learn about more advanced options using the rest of the documentation. Here are some examples of topics you may be interested in exploring as you continue: - :ref:`Writing a Javascript function to generate your study protocol` @@ -37,10 +38,10 @@ Although we hope this tutorial has provided a solid introduction to designing an 3. So what comes next? --------------------------------------- -Now that you have a basic understanding of how to use Lookit, you're ready to start setting up the study you want to run! One frequent question is how long this will take. Once you have your study completely planned out--the design, instructions, and stimuli all ready--it's likely to be very quick, perhaps a few days to implement. But the (unanimous) experience of researchers who have beta-tested Lookit is that it takes much longer than expected for purely non-technical reasons: the process of fleshing out stimuli and instructions often brings up substantive issues about experimental design that might not have been caught as early when piloting in the lab. +Now that you have a basic understanding of how to use CHS, you're ready to start setting up the study you want to run! One frequent question is how long this will take. Once you have your study completely planned out--the design, instructions, and stimuli all ready--it's likely to be very quick, perhaps a few days to implement. But the (unanimous) experience of researchers who have beta-tested Lookit is that it takes much longer than expected for purely non-technical reasons: the process of fleshing out stimuli and instructions often brings up substantive issues about experimental design that might not have been caught as early when piloting in the lab. In addition to the technical preparations you're making to be able to implement your study, you'll want to make sure you're ready to go from a legal and IRB standpoint! -See :ref:`IRB and legal info ` for details. Basically, though, you'll need to get an institutional agreement signed; complete a short quiz about the Terms of Use because otherwise no one reads them; and include data collection on Lookit in your IRB protocol. +See :ref:`IRB and legal info ` for details. Basically, though, you'll need to get an institutional agreement signed; complete a short quiz about the Terms of Use because otherwise no one reads them; and include data collection on CHS in your IRB protocol. You can go ahead and:ref:`create a lab` at any point, and it'll be approved once you've done the steps above. diff --git a/docs/source/tutorial-protocol-exercises.rst b/docs/source/tutorial-protocol-exercises.rst index 4286c20f..fd194d17 100644 --- a/docs/source/tutorial-protocol-exercises.rst +++ b/docs/source/tutorial-protocol-exercises.rst @@ -1,5 +1,5 @@ ########################################## -5. Study protocol exercises +5. Lookit study protocol exercises ########################################## Now that you've walked through the process of creating your own studies, it's time to try out what you've learned! At the end of this section, you will: @@ -123,13 +123,13 @@ You'd like to include a training section where parents are asked a multiple-choi 7. Draft a parent-facing study description ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Write a :ref:`purpose` field for a Lookit study. The study can be: +Write a :ref:`purpose` field for a CHS study. The study can be: -* Something you're planning to run on Lookit +* Something you're planning to run on CHS * Something you've run in your lab before, or a favorite study from another lab that you're familiar with * Or if you don't have anything in mind: try writing a purpose field for a study about `infant-directed speech preference `_. -Post your draft description of the study purpose in the Lookit Slack workspace (#researchers) for feedback (yes, really!). +Post your draft description of the study purpose in the Lookit/CHS Slack workspace (#researchers) for feedback (yes, really!). Look for other tutorial participants' drafts to give feedback on too! As you read, consider: diff --git a/docs/source/tutorial-second-study.rst b/docs/source/tutorial-second-study.rst index ed70b161..087577bc 100644 --- a/docs/source/tutorial-second-study.rst +++ b/docs/source/tutorial-second-study.rst @@ -17,16 +17,16 @@ Introduction: intermodal matching study Imagine you're looking to replicate the finding that infants can detect which moving face "goes with" a speech stream, an ability known as intermodal matching. In your study, babies will watch several short videos of two women's faces - one face on the left and one on the right. Both women are talking, but babies only hear the audio from one of them in each clip. (There are four trials, and who's talking is counterbalanced - babies either hear left, right, right, left audio or right, left, left, right audio.) -You plan to code the video collected on Lookit for preferential looking - whether the child is looking to the left of the screen, right of the screen, or away. Because you are eventually hoping to develop a measure that can be used to detect individual differences linked to social development, you are also including a short survey on parenting beliefs. +You plan to code the video collected on CHS for preferential looking - whether the child is looking to the left of the screen, right of the screen, or away. Because you are eventually hoping to develop a measure that can be used to detect individual differences linked to social development, you are also including a short survey on parenting beliefs. -.. admonition:: This is a real study that was run on Lookit! +.. admonition:: This is a real study that was run on CHS! The study stimuli and protocol have been generously shared by Halie Olson and Rebecca Saxe for use as an example. Slight modifications have been made for teaching purposes. Creating the study and filling out study fields ----------------------------------------------- -This time, instead of copying an existing study, we're going to create our own from scratch so that we see every field. Go to ``_ and click the green "Create Study" button at top right: +This time, instead of copying an existing study, we're going to create our own from scratch so that we see every field. Go to ``_ and click the green "Create Study" button at top right: .. image:: _static/img/tutorial/create_study_button.png :alt: Create Study button @@ -60,7 +60,7 @@ Compensation Here's the actual compensation description that was used - you can copy it: "After you participate, we'll email you a $4 Amazon gift card as a thank-you. (One gift card per child; child must be in the age range for the study.)" Exit URL - After the study let's send families to their study history, where they can see their videos right away! Use "https://lookit.mit.edu/studies/history/" + After the study let's send families to their study history, where they can see their videos right away! Use "https://childrenhelpingscience.com/studies/history/" Participant Eligibility Description "For babies ages 4-18 months" @@ -126,7 +126,7 @@ That defines a frame that's now available for us to use. In order to actually us .. admonition:: The key for your frame can be whatever you want - There's nothing magical about the "video-config" key given to this frame - you can change it to whatever you want, as long as the key in ``frames`` matches what you call it in ``sequence``. Just don't put an underscore in it (see the :ref:`experiment runner documentation `). + There's nothing magical about the "video-config" key given to this frame - you can change it to whatever you want, as long as the key in ``frames`` matches what you call it in ``sequence``. Just don't put an underscore in it (see the :ref:`Lookit experiment runner documentation `). Close the editor, save your protocol, and preview your study. Make sure to click "build experimenter runner" if you have not already done so; you will be emailed when it is built and then you can preview your study. You should see the setup frame, looking just like the screenshot in the docs. @@ -135,7 +135,7 @@ Close the editor, save your protocol, and preview your study. Make sure to click Now that your participants have their webcam set up, the very first thing you need to do - before starting any study procedures - is collect informed consent. Consent frames are treated somewhat specially: you will only see any data from participants who get through your consent page, and videos collected on the consent page will be available for you to review and confirm before you can access the remaining data from the corresponding sessions. -Unless you receive specific permission from Lookit, you'll be asked to use the standard video consent (and/or assent) frames to keep the experience for participants consistent. +Unless you receive specific permission from CHS, you'll be asked to use the standard video consent (and/or assent) frames to keep the experience for participants consistent. This study is for babies, so we don't need to collect child assent, just parental consent. Go to the frame docs and select 'exp-lookit-video-consent' on the sidebar. Just like you did for the setup frame, copy the sample frame definition (``"video-consent": {...}``) and add it to your study protocol frames and sequence, like this: @@ -482,7 +482,7 @@ It will have the same format and possible answers as the others. You can copy an 8. Exit survey ~~~~~~~~~~~~~~ -Finally, to wrap up our study we need to include an "exp-lookit-exit-survey" frame. (This is required of all Lookit studies to keep the experience for parents fairly consistent.) This is where parents have an option to choose how you may share their video, if at all, and to give you some feedback if they want to. It's also where you'll provide some "debriefing" information, just like you might when chatting with the family after they came into the lab. There are more guidelines about what your debriefing should contain under :ref:`the sample study outline `. +Finally, to wrap up our study we need to include an "exp-lookit-exit-survey" frame. (This is required of all internal CHS studies to keep the experience for parents fairly consistent.) This is where parents have an option to choose how you may share their video, if at all, and to give you some feedback if they want to. It's also where you'll provide some "debriefing" information, just like you might when chatting with the family after they came into the lab. There are more guidelines about what your debriefing should contain under :ref:`the sample study outline `. You guessed it - copy and paste the frame below into ``frames`` in your protocol, and add "exit-survey" to your ``sequence``. Put the frames in your ``sequence`` in order and try out the entire study! @@ -918,10 +918,10 @@ It is HARD, for instance, to write a few-sentence "elevator pitch" for your stud It's also very hard to condense text instructions into something concise, non-condescending, and complete. (The examples above aren't perfect!) You may realize there's more than you thought to explain about how to do your study (e.g. how to avoid biasing the child), and that you want to add some training trials with feedback, video instructions, or more detailed audio instructions. -So this is a general note of caution: yes, in some respects it's easy to "throw a study up on Lookit." (Or at least we're trying to make it easy!) But it will likely take you longer than you expect to go from "We know exactly how we want our study to work" to "We're up and running," in large part because of these sorts of details. And it is absolutely worth putting in the time to come up with a study protocol that doesn't just "work" but is clear and easy to follow for parents - not least because we're all sharing the same subject pool and reputation as a fun place to do studies. +So this is a general note of caution: yes, in some respects it's easy to "throw a study up on CHS." (Or at least we're trying to make it easy!) But it will likely take you longer than you expect to go from "We know exactly how we want our study to work" to "We're up and running," in large part because of these sorts of details. And it is absolutely worth putting in the time to come up with a study protocol that doesn't just "work" but is clear and easy to follow for parents - not least because we're all sharing the same subject pool and reputation as a fun place to do studies. Using the documentation to learn about more advanced features -------------------------------------------------------------- -We hope that working through some examples has been helpful, but the Lookit documentation goes beyond just the tutorial! You can explore using the sidebar on the left to view detailed guides to preparing your study (including advanced topics not covered in this tutorial), managing your data, and developing your own custom frames. We recommend using the search function within the documentation, which ensures your results come only from the current, up-to-date version of the docs, rather than any archived older versions that might pop up on Google. +We hope that working through some examples has been helpful, but the CHS documentation goes beyond just the tutorial! You can explore using the sidebar on the left to view detailed guides to preparing your study (including advanced topics not covered in this tutorial), managing your data, and developing your own custom frames. We recommend using the search function within the documentation, which ensures your results come only from the current, up-to-date version of the docs, rather than any archived older versions that might pop up on Google. diff --git a/docs/source/vision-for-lookit.rst b/docs/source/vision-for-lookit.rst index 7a282283..73349015 100644 --- a/docs/source/vision-for-lookit.rst +++ b/docs/source/vision-for-lookit.rst @@ -1,13 +1,13 @@ Project overview ========================== -Lookit is a platform for doing developmental research run by a small academic group at MIT. +Children Helping Science (CHS) is a platform for doing developmental research run by a small academic group at MIT. Our vision is of **a collaborative online lab**: - Large collaborative “online lab” run by group at MIT - Single participant interface; shared participant pool, servers; central & shared recruitment in addition to individual labs’ efforts -- Lookit team provides training, IRB coordination, study +- CHS team provides training, IRB coordination, study implementation, design support, ongoing development, documentation - Researchers independent, but with centralized approval of studies (covering technical problems, basic compliance with ethical diff --git a/snippets.rst b/snippets.rst index e9f4ea21..bd9239e3 100644 --- a/snippets.rst +++ b/snippets.rst @@ -8,7 +8,7 @@ .. admonition:: Where's the code I need? - If you only want to change something about the Lookit site, without touching experiment functionality... + If you only want to change something about the CHS site, without touching experiment functionality... @@ -17,4 +17,4 @@ `the wiki `_ -`Lookit platform milestones `_ +`CHS platform milestones `_ From 9c3e2c6934da439b5faafac04aa675c861784524 Mon Sep 17 00:00:00 2001 From: Becky Gilbert Date: Tue, 5 Nov 2024 09:25:45 -0800 Subject: [PATCH 02/24] move tutorial pages into separate lookit tutorial section --- docs/source/tutorial-lookit/index.rst | 9 +++ .../tutorial-first-study.rst | 80 +++++++++---------- .../tutorial-protocol-exercises.rst | 2 +- .../tutorial-second-study.rst | 18 ++--- 4 files changed, 59 insertions(+), 50 deletions(-) create mode 100644 docs/source/tutorial-lookit/index.rst rename docs/source/{ => tutorial-lookit}/tutorial-first-study.rst (93%) rename docs/source/{ => tutorial-lookit}/tutorial-protocol-exercises.rst (99%) rename docs/source/{ => tutorial-lookit}/tutorial-second-study.rst (98%) diff --git a/docs/source/tutorial-lookit/index.rst b/docs/source/tutorial-lookit/index.rst new file mode 100644 index 00000000..858123d3 --- /dev/null +++ b/docs/source/tutorial-lookit/index.rst @@ -0,0 +1,9 @@ +Create a Lookit study +============================= + +.. toctree:: + :maxdepth: 1 + + tutorial-first-study + tutorial-second-study + tutorial-protocol-exercises \ No newline at end of file diff --git a/docs/source/tutorial-first-study.rst b/docs/source/tutorial-lookit/tutorial-first-study.rst similarity index 93% rename from docs/source/tutorial-first-study.rst rename to docs/source/tutorial-lookit/tutorial-first-study.rst index 05148be8..ead77813 100644 --- a/docs/source/tutorial-first-study.rst +++ b/docs/source/tutorial-lookit/tutorial-first-study.rst @@ -1,10 +1,10 @@ -###################################### -3. Setting up your first Lookit study -###################################### +######################################### +1. Setting up your first Lookit study +######################################### In this section, you will be creating your first Lookit study. You will learn how to find and use experiment components, specify your protocol, and test out and troubleshoot your study. -This section is designed for researchers planning to build an internal Lookit study. If your first study will be written with jsPsych, then you can see the :ref:`jspsych tutorial `. Or if your study involves a link to an experiment hosted somewhere else, skip to part 6 of this tutorial. +This section is designed for researchers planning to build an internal Lookit study. If your first study will be written with jsPsych, then you can see the :ref:`jspsych tutorial `. Or if your study involves a link to an experiment hosted somewhere else, skip to part 6 of this tutorial. Step 1: Clone the 'Lookit tutorial part 1' study ------------------------------------------------- @@ -13,47 +13,47 @@ To get started, log in to CHS as an experimenter. (Go to ``. For now, we'll just change the name of the study and the thumbnail image that's displayed to participants. Rename your study to "[Your name]'s awesome tutorial study" and upload a different thumbnail image: -.. image:: _static/img/tutorial/study_edit_name.png +.. image:: ../_static/img/tutorial/study_edit_name.png :alt: Study edit view - changing name and image Then click "Save Changes" down below: -.. image:: _static/img/tutorial/study_edit_save.png +.. image:: ../_static/img/tutorial/study_edit_save.png :alt: Study edit save button After saving, click the name of your study to return to the study detail view: -.. image:: _static/img/tutorial/return_to_study_detail.png +.. image:: ../_static/img/tutorial/return_to_study_detail.png :alt: Link back to study detail view You should see your changes reflected, like this: -.. image:: _static/img/tutorial/study_after_save.png +.. image:: ../_static/img/tutorial/study_after_save.png :alt: Study detail view after changes Congratulations! You've created and edited your first study. @@ -67,12 +67,12 @@ You may have noticed that below your thumbnail and basic study info, there's a s This section also shows whether your "experiment runner" is "built" yet. You should see a bar like this: -.. image:: _static/img/tutorial/dependencies_not_built.png +.. image:: ../_static/img/tutorial/dependencies_not_built.png :alt: Dependency status area Click the "Build experiment runner" button. You should see a notification at the top of the screen, something like this: -.. image:: _static/img/tutorial/scheduled_for_preview.png +.. image:: ../_static/img/tutorial/scheduled_for_preview.png :alt: Scheduled for build notification What is this "experiment runner"? When you create a Lookit study, you specify what types of pages or "frames" to use, and provide parameters for each - for example you supply the text for an instructions page, videos or images to show in a preferential looking trial, audio and images for a storybook page, and so on. The Lookit frameplayer interprets this information and turns it into an interactive study families can participate in. There's code behind the scenes, which you don't have to deal with, to handle that interpretation and to make each page "go" (saying what each button should do and what data to collect, arranging and starting/stopping video, etc.) Rather than all studies sharing that code, each study gets its own siloed little environment called a Docker image where it will run. @@ -163,12 +163,12 @@ Here are several things that are almost, but not quite, valid JSON objects. Copy OK, congrats on learning all about JSON! Your study should be ready to preview by now. You should have an email in your inbox from CHS about this, and if you refresh the page you're on, you should see something like this: -.. image:: _static/img/tutorial/preview_built.png +.. image:: ../_static/img/tutorial/preview_built.png :alt: Preview built status display Click on "Preview study" near the top of the page: -.. image:: _static/img/tutorial/preview_button.png +.. image:: ../_static/img/tutorial/preview_button.png :alt: Preview button This will take you to a "study detail" page just like the one participants see when they click on a study at childrenhelpingscience.com/studies. You will need to register at least one child and respond to the demographic survey (you don't need to respond to all questions, or use real information). Then you will be able to click "Preview now" to proceed through the study as a participant. It's a rough, abbreviated implementation of one condition from `Schulz, Bonawitz, and Griffiths (2007) `_ - you'll read through a storybook about Bunny, who sometimes gets a tummyache, and eventually answer a question about what makes her tummy hurt. @@ -183,27 +183,27 @@ Step 3: Get comfortable making changes to how your study works The "meat" of your study is in the "Study protocol configuration", which you can change from the Edit Study view. Scroll down and click on the text here: -.. image:: _static/img/tutorial/click_to_edit_json.png +.. image:: ../_static/img/tutorial/click_to_edit_json.png :alt: JSON as shown on study edit view This opens up an editor. It will show a scary amount of text like this: -.. image:: _static/img/tutorial/ugly_json.png +.. image:: ../_static/img/tutorial/ugly_json.png :alt: Unformatted JSON Click the "Beautify" button to format it nicely: -.. image:: _static/img/tutorial/pretty_json.png +.. image:: ../_static/img/tutorial/pretty_json.png :alt: Formatted JSON This whole "protocol" is a JSON document, like we learned about while you were waiting for your preview dependencies to build. Using the triangles on the left may help you to explore and understand its structure better. Try collapsing headers by clicking those triangles until you can see this overall structure: -.. image:: _static/img/tutorial/collapsed.png +.. image:: ../_static/img/tutorial/collapsed.png :alt: Formatted JSON collapsed into high level keys There are two high-level keys: ``frames`` and ``sequence``. The ``sequence`` list says what frames should be in your study, in what order. The ``frames`` object is like a dictionary where the Lookit frameplayer will go to understand what each frame in the sequence should be like. Notice that each of the strings in the ``sequence`` is itself a key in ``frames``. For instance, the sequence starts with ``video-config``. We can expand the ``video-config`` key in ``frames`` to see more about that frame: -.. image:: _static/img/tutorial/video_config_expanded.png +.. image:: ../_static/img/tutorial/video_config_expanded.png :alt: Formatted JSON collapsed into high level keys with one frame expanded Making a change to the sequence @@ -223,28 +223,28 @@ Let's change the ``sequence``` to see how it affects the study. In the editor, m Now click "Close" at the top right to exit the editor: -.. image:: _static/img/tutorial/click_to_close.png +.. image:: ../_static/img/tutorial/click_to_close.png :alt: Where to click to close editor This will return you to the Edit Study page, but your changes aren't saved yet. Scroll down and click "Save Changes": -.. image:: _static/img/tutorial/click_to_save_json.png +.. image:: ../_static/img/tutorial/click_to_save_json.png :alt: Where to click to save JSON You should see a message at the top like the following. (If not, click on your protocol again and resolve any problems that are preventing it from saving.) -.. image:: _static/img/tutorial/changes_saved.png +.. image:: ../_static/img/tutorial/changes_saved.png :alt: Changes saved message Now you can preview your edited study by clicking on the blue "Preview" button again: -.. image:: _static/img/tutorial/preview_button.png +.. image:: ../_static/img/tutorial/preview_button.png :alt: Preview button This time, you should be right at the instructions, instead of starting with the video configuration frame! If you back and click "Preview study" and then "Preview now" again, you should go right away to a page like this: -.. image:: _static/img/tutorial/instructions_page.png +.. image:: ../_static/img/tutorial/instructions_page.png :alt: Instructions frame That's because we moved the "instructions" frame to the start of our sequence. If you want to make changes to a particular frame, sticking it at the beginning of your sequence can make it easier to rapidly view your changes as you make them. @@ -394,27 +394,27 @@ Click to preview your study, and from that browser window/tab, let's get your we **If you're using Firefox**: Click the "hamburger menu" (three horizontal lines) in the top right corner of your browser and click "Web Developer" (yep, that's you now!): -.. image:: _static/img/tutorial/firefox_web_developer.png +.. image:: ../_static/img/tutorial/firefox_web_developer.png :alt: Firefox hamburger menu Click "Web Console": -.. image:: _static/img/tutorial/firefox_dev_menu.png +.. image:: ../_static/img/tutorial/firefox_dev_menu.png :alt: Firefox web developer menu And you should see something like this: -.. image:: _static/img/tutorial/firefox_console.png +.. image:: ../_static/img/tutorial/firefox_console.png :alt: Firefox console example **If you're using Chrome**: Click the three dots in the upper right corner, then "More Tools," then "Developer Tools": -.. image:: _static/img/tutorial/chrome_dev_tools_menu.png +.. image:: ../_static/img/tutorial/chrome_dev_tools_menu.png :alt: Chrome developer tools menu You should see something like this: -.. image:: _static/img/tutorial/chrome_console.png +.. image:: ../_static/img/tutorial/chrome_console.png :alt: Chrome console example .. admonition:: Browser-dependent behavior @@ -429,7 +429,7 @@ Now that you've gotten your web console open, you'll see a bunch of information * On the staging server there is a known bug that the fontawesome library doesn't load properly (but it does on "production" - the real CHS site) - so you may see some errors that a resource failed to load properly, like this: -.. image:: _static/img/tutorial/fa_error.png +.. image:: ../_static/img/tutorial/fa_error.png :alt: Fontawesome error example Leave your preview tab open, and return to the browser tab where you have the "Edit Study " page open. Let's deliberately introduce a problem in our study JSON and see what we can learn from the preview. Try adding something to the "sequence" without defining it in "frames," like this: @@ -447,7 +447,7 @@ Leave your preview tab open, and return to the browser tab where you have the "E Close, save changes, and then return to your preview tab and refresh it. You'll see a totally blank page, which would be very confusing if you didn't know what had gone wrong! But if you look down at the web console, you should see an error like this: -.. image:: _static/img/tutorial/sequence_error.png +.. image:: ../_static/img/tutorial/sequence_error.png :alt: Example sequence error This explains that the problem is that the Lookit frameplayer can't make sense of your study JSON, because it doesn't have a "definition" available in the "frames" value for the frame "new-and-exciting-page" that you added to your sequence. @@ -472,7 +472,7 @@ Return to the study edit page and open up the JSON editor again. Remove that "ne Try deleting one of these lines, like ``"PIContact": "Jane Smith at (123) 456-7890",``. Close, save, and refresh your preview. Once you get to the consent page, you should see an error like this complaining about the missing parameter: -.. image:: _static/img/tutorial/missing_parameter.png +.. image:: ../_static/img/tutorial/missing_parameter.png :alt: Example missing parameter error Note that this doesn't stop the frame from working at all (that bit of text is just missing from the consent form if you look carefully) - but this sort of error can be a useful clue in more complicated situations! @@ -486,12 +486,12 @@ You may have noticed when you tried out the study that the ending was a little a Open the protocol JSON editor again. Inside the ``frames`` object, find the ``storybook-causal`` frame definiton. It should look something like this (with the long ``frameList`` collapsed): -.. image:: _static/img/tutorial/storybook_causal_json.png +.. image:: ../_static/img/tutorial/storybook_causal_json.png :alt: Storybook JSON displayed This is a frame "group" that actually bundles together a list of frames, adding some ``commonFrameProperties`` to each one. You don't have to understand that yet! For now, let's take a look inside the ``frameList`` where the list of storybook pages is. Each element of this list is an object with ``images`` and ``audioSources`` - here's what it looks like collapsing most of those list elements: -.. image:: _static/img/tutorial/list_of_pages.png +.. image:: ../_static/img/tutorial/list_of_pages.png :alt: Storybook pages JSON We're just going to add one more page to the end. Within the ``frameList`` list, after the last object, add a comma and then the following: @@ -536,17 +536,17 @@ In addition to this tutorial and the information in this documentation about how Let's take a look at the ``exp-lookit-images-audio`` documentation to see what options we have. Find it on the left sidebar and click on it. Here's what you'll see: -.. image:: _static/img/tutorial/frame_doc_1.png +.. image:: ../_static/img/tutorial/frame_doc_1.png :alt: Frame documentation page - top Each frame documentation page has the same sections you can use to learn more about how to customize it, what data it collects, and so on. Near the top under "What it looks like", you can see a screenshot of the frame (or a collection of example screenshots). Under "Example" you'll find an example that you can generally copy and paste to get started: -.. image:: _static/img/tutorial/frame_doc_2.png +.. image:: ../_static/img/tutorial/frame_doc_2.png :alt: Annotated frame documentation page - examples The final three sections are "Parameters," "Data collected," and "Events recorded." -.. image:: _static/img/tutorial/frame_doc_3.png +.. image:: ../_static/img/tutorial/frame_doc_3.png :alt: Annotated frame documentation page - methods, properties, events Click on "Parameters" to see all the properties we can add to the frame definition in our protocol. You'll see that some of the things we can set are "audio," "autoProceed," "doRecording," "durationSeconds," "images," "parentTextBlock," and "showProgressBar." Each one includes an explanation of what it does and what format its value needs to be in. diff --git a/docs/source/tutorial-protocol-exercises.rst b/docs/source/tutorial-lookit/tutorial-protocol-exercises.rst similarity index 99% rename from docs/source/tutorial-protocol-exercises.rst rename to docs/source/tutorial-lookit/tutorial-protocol-exercises.rst index fd194d17..0de5bf7a 100644 --- a/docs/source/tutorial-protocol-exercises.rst +++ b/docs/source/tutorial-lookit/tutorial-protocol-exercises.rst @@ -1,5 +1,5 @@ ########################################## -5. Lookit study protocol exercises +3. Lookit study protocol exercises ########################################## Now that you've walked through the process of creating your own studies, it's time to try out what you've learned! At the end of this section, you will: diff --git a/docs/source/tutorial-second-study.rst b/docs/source/tutorial-lookit/tutorial-second-study.rst similarity index 98% rename from docs/source/tutorial-second-study.rst rename to docs/source/tutorial-lookit/tutorial-second-study.rst index 087577bc..77ad7f3b 100644 --- a/docs/source/tutorial-second-study.rst +++ b/docs/source/tutorial-lookit/tutorial-second-study.rst @@ -1,5 +1,5 @@ ##################################################### -4. A study from the ground up +2. A Lookit study from the ground up ##################################################### Now that you've gotten your feet wet and are comfortable using Lookit's experimenter interface to modify your study protocol configuration, it's time to take a closer look at how to build your own study. @@ -28,12 +28,12 @@ Creating the study and filling out study fields This time, instead of copying an existing study, we're going to create our own from scratch so that we see every field. Go to ``_ and click the green "Create Study" button at top right: -.. image:: _static/img/tutorial/create_study_button.png +.. image:: ../_static/img/tutorial/create_study_button.png :alt: Create Study button This will bring you to a screen with a bunch of fields to fill out: -.. image:: _static/img/tutorial/create_study.png +.. image:: ../_static/img/tutorial/create_study.png :alt: Study creation view In a separate tab, open up the documentation about these fields for fuller explanations of what each one should contain: :ref:`study fields`. Below is the study-specific information you'll need to fill out each field. @@ -109,19 +109,19 @@ We'll start with a standard setup frame called "exp-video-config". Take a look at the documentation for this frame :ref:`here `. You'll see a screenshot of what it looks like, and under "Examples" you'll see examples of how to define this frame in your study protocol: -.. image:: _static/img/tutorial/exp_video_config.png +.. image:: ../_static/img/tutorial/exp_video_config.png :alt: Exp-video-config frame docs Copy one of the definitions of the "video-config" frame (``"video-config": { ... }``, as shown highlighted above), and open up your study's protocol editor. Paste this into the "frames" value, like this: -.. image:: _static/img/tutorial/video_config_added_to_frames.png +.. image:: ../_static/img/tutorial/video_config_added_to_frames.png :alt: Adding the video-config example to frames Edit the text if you'd like, so it references your own lab and an appropriate contact method! That defines a frame that's now available for us to use. In order to actually use it, add it to your "sequence" as well: -.. image:: _static/img/tutorial/video_config_added_to_sequence.png +.. image:: ../_static/img/tutorial/video_config_added_to_sequence.png :alt: Adding the video-config example to sequence .. admonition:: The key for your frame can be whatever you want @@ -139,14 +139,14 @@ Unless you receive specific permission from CHS, you'll be asked to use the stan This study is for babies, so we don't need to collect child assent, just parental consent. Go to the frame docs and select 'exp-lookit-video-consent' on the sidebar. Just like you did for the setup frame, copy the sample frame definition (``"video-consent": {...}``) and add it to your study protocol frames and sequence, like this: -.. image:: _static/img/tutorial/adding_video_consent.png +.. image:: ../_static/img/tutorial/adding_video_consent.png :alt: Adding the video-consent example to sequence Note that you will need to add commas between the previous and new items in both "frames" and "sequence", as circled above. Save and preview again. Now when you click "Next" from the video config page, you'll see a consent page. The sample text is pretty silly, though! Change each of the following fields to more appropriate text for this study, substituting in your own information for the things shown in ````. For more information on what each of these fields is, click on "Parameters" in the frame documentation: -.. image:: _static/img/tutorial/frame_docs_properties.png +.. image:: ../_static/img/tutorial/frame_docs_properties.png :alt: Properties as displayed in frame docs PIName @@ -196,7 +196,7 @@ Go to the frame documentation, and select the "exp-lookit-text" frame. Just like For convenience, this time, let's put "study-intro" FIRST in the sequence, so that when we preview our study it's easy for us to see the changes we make to customize the text on this frame: -.. image:: _static/img/tutorial/study_intro_first.png +.. image:: ../_static/img/tutorial/study_intro_first.png :alt: Putting the study-intro frame first Save your protocol and go ahead and preview your study. You should see a simple text frame first. Let's change the ``blocks`` value to show an appropriate overview for this study: copy and paste the section below to replace the existing ``"blocks": [...]`` piece: From 4b2f4814feabbedf37ce3b6c5c844225805c5e36 Mon Sep 17 00:00:00 2001 From: Becky Gilbert Date: Thu, 7 Nov 2024 11:41:26 -0800 Subject: [PATCH 03/24] add jsPsych tutorial pages nested in tutorial step 3 --- docs/source/tutorial-jspsych/index.rst | 7 + .../tutorial-jspsych-first-study.rst | 391 ++++++++++++++++++ 2 files changed, 398 insertions(+) create mode 100644 docs/source/tutorial-jspsych/index.rst create mode 100644 docs/source/tutorial-jspsych/tutorial-jspsych-first-study.rst diff --git a/docs/source/tutorial-jspsych/index.rst b/docs/source/tutorial-jspsych/index.rst new file mode 100644 index 00000000..f1fb410b --- /dev/null +++ b/docs/source/tutorial-jspsych/index.rst @@ -0,0 +1,7 @@ +Create a jsPsych study +======================= + +.. toctree:: + :maxdepth: 1 + + tutorial-jspsych-first-study \ No newline at end of file diff --git a/docs/source/tutorial-jspsych/tutorial-jspsych-first-study.rst b/docs/source/tutorial-jspsych/tutorial-jspsych-first-study.rst new file mode 100644 index 00000000..ace4629a --- /dev/null +++ b/docs/source/tutorial-jspsych/tutorial-jspsych-first-study.rst @@ -0,0 +1,391 @@ +.. _jspsych-tutorial-first-study: + +##################################################### +1. Setting up your first jsPsych study +##################################################### + +In this section, you will be creating your first jsPsych study on CHS! + +.. admonition:: Who should do this section? + + **This section is designed for researchers planning to build an internal jsPsych study that will be hosted on CHS.** If your first study will be written with the Lookit experiment runner (ember-lookit-frameplayer), then you can do the :ref:`Lookit section ` instead for this "Create a study" step in the tutorial. Or, if your study involves a link to an experiment hosted somewhere else (i.e. an "external" study), you can skip to step 4 in the tutorial: :ref:`Managing Study Data `. + +`jsPsych `__ is a comprehensive open source library for creating behavioral experiments that run in a web browser. The CHS team has created a small library of custom jsPsych plugins/extensions to "fill in the gaps" in the sort of functionality CHS researchers need for child development studies. This functionality includes: standardized webcam/mic configuration steps, video-recorded consent, trial/session recording, and standardized exit surveys. Therefore jsPsych studies on CHS will typically use a mix of: + +1. Our `customized CHS jsPsych plugins and extensions `__, which were created specifically for the CHS platform, and +2. The "official" jsPsych `plugins `__ and `extensions `__, which are more general-purpose and fully-featured for creating a wide variety of experiments. + +This is intended to be a "quick start" tutorial, so there's little explanation about how things work, why they're written out this way, or what options you have when creating experiments/trials. If you're interested in learning more as you go along, you can always reference the documentation links above. + +Step 1: Create a new CHS jsPsych study +------------------------------------------------- + +Your first step is to create a new CHS study and fill in some details on the study creation form. It doesn't matter what you put here, but you will need some placeholder text/values in most fields to be able to create a study successfully. When you get to the bottom of the form, under "Study Type", choose the "jsPsych" option. + +Step 2: Add a Hello World trial +------------------------------------------------- + +Now you should see the Edit Study Design page, with a box that says "jsPsych Experiment Code". Click in that box - it should open into a larger window. This is your JavaScript editor, where you will put all of the code you need to create your experiment. + +.. admonition:: What about the HTML? + + If you've used jsPsych before, you might be used to creating an HTML file (e.g. in Step 1 of this `jsPsych tutorial `__). On CHS, we've created the HTML file and loaded the jsPsych packages for you, so all you need to do is write the JavaScript. + +Let's start by initializing jsPsych with the ``initJsPsych`` function: + +.. code:: javascript + + const jsPsych = initJsPsych(); + +Then create a Hello World trial using jsPsych's `HTML button response plugin `__: + +.. code:: javascript + + const hello_trial = { + type: jsPsychHtmlButtonResponse, + stimulus: 'Hello world!', + choices: ['Next'] + }; + +Now that we're done creating trials (for now), we can tell jsPsych to run our experiment. We need to use the ``jsPsych.run`` function, and give it the array of trial objects that we want to run. In this case, we're only running one trial, so it looks like this: + +.. code:: javascript + + jsPsych.run([hello_trial]); + + +Here's the full code so far: + +.. code:: javascript + + const jsPsych = initJsPsych(); + + const hello_trial = { + type: jsPsychHtmlButtonResponse, + stimulus: 'Hello world!', + choices: ['Next'] + }; + + jsPsych.run([hello_trial]); + + +Click "Close" to close the JavaScript editor (your code will still be there), then click "Save Changes". On your experiment details page, click "Preview" to run through your experiment. + + +Step 3: Add a video config trial +------------------------------------------------- + +That was fun! But this isn't a very realistic or useful experiment. Now let's add a video configuration trial. This trial type is used for setting up the participant's webcam and mic. It is *required* before doing any kind of webcam display or recording during the experiment, so most experiments will include this trial early on in the jsPsych "timeline" (trial sequence). + +Let's create a "video_config" trial using the custom `CHS video config plugin `__. + +.. code:: javascript + + const video_config = { type: chsRecord.VideoConfigPlugin }; + + +We'll add this trial to the ``jsPsych.run`` trial sequence before the hello world trial. + +.. code:: javascript + + jsPsych.run([video_config, hello_trial]); + + +Here's what we have now: + +.. code:: javascript + + const jsPsych = initJsPsych(); + + const video_config = { type: chsRecord.VideoConfigPlugin }; + + const hello_trial = { + type: jsPsychHtmlButtonResponse, + stimulus: 'Hello world!', + choices: ['Next'] + }; + + jsPsych.run([video_config, hello_trial]); + + +When you save your code and preview the experiment, you should see the video config trial first, followed by the hello world trial. + + +Step 4: Add a video consent trial +------------------------------------------------- + +Like the video config trial, CHS studies almost always include a video consent trial. Let's add that after the video config trial. + +The video consent trial uses a custom CHS plugin. It requires more parameters than we've needed so far, because this trial must contain a number of details about the experiment. Here we're adding a video consent trial with all of the required parameters, but there are some optional parameters you can change as well. See our CHS jsPsych documentation about the CHS Record package's `VideoConsentPlugin `__ for more information about the required and optional parameters. + +Here's the full code, including the new video consent trial: + +.. code:: javascript + + const jsPsych = initJsPsych(); + + const video_config = { type: chsRecord.VideoConfigPlugin }; + + const video_consent = { + type: chsRecord.VideoConsentPlugin, + PIName: "Jane Smith", + institution: "Science University", + PIContact: "Jane Smith at 123 456 7890", + payment: "After you finish the study, we will email you a $5 BabyStore gift card within approximately three days. To be eligible for the gift card your child must be in the age range for this study, you need to submit a valid consent statement, and we need to see that there is a child with you. But we will send a gift card even if you do not finish the whole study or we are not able to use your child's data! There are no other direct benefits to you or your child from participating, but we hope you will enjoy the experience.", + procedures: "Your child will be shown pictures of lots of different cats, along with noises that cats make like meowing and purring. We are interested in which pictures and sounds make your child smile. We will ask you (the parent) to turn around to avoid influencing your child's responses.", + purpose: "Why do babies love cats? This study will help us find out whether babies love cats because of their soft fur or their twitchy tails." + }; + + const hello_trial = { + type: jsPsychHtmlButtonResponse, + stimulus: 'Hello world!', + choices: ['Next'] + }; + + jsPsych.run([video_config, video_consent, hello_trial]); + + +Now when you save your code and preview the experiment, you should see: a video config trial, a video consent trial, and the hello world trial. + + +Step 5: Add an exit survey trial +------------------------------------------------- + +As with the video config and consent trials, CHS studies will almost always need an "exit survey" trial at the end. This helps create a more consistent experience for CHS families, and it ensures they always have a chance to give feedback or withdraw their data. It also marks the response as "complete" in our database, so that you can access the data. + +Let's create the exit survey trial using a custom CHS jsPsych plugin. Again, we're just using the default settings, but there are also some optional parameters you can set - see the CHS Surveys package `ExitSurveyPlugin `__ documentation for more information. + +.. code:: javascript + + const exit_survey = { type: chsSurvey.ExitSurveyPlugin }; + + +This trial will go at the very end of our experiment timeline. + +.. code:: javascript + + jsPsych.run([video_config, video_consent, hello_trial, exit_survey]); + + +The full code so far: + +.. code:: javascript + + const jsPsych = initJsPsych(); + + const video_config = { type: chsRecord.VideoConfigPlugin }; + + const video_consent = { + type: chsRecord.VideoConsentPlugin, + PIName: "Jane Smith", + institution: "Science University", + PIContact: "Jane Smith at 123 456 7890", + payment: "After you finish the study, we will email you a $5 BabyStore gift card within approximately three days. To be eligible for the gift card your child must be in the age range for this study, you need to submit a valid consent statement, and we need to see that there is a child with you. But we will send a gift card even if you do not finish the whole study or we are not able to use your child's data! There are no other direct benefits to you or your child from participating, but we hope you will enjoy the experience.", + procedures: "Your child will be shown pictures of lots of different cats, along with noises that cats make like meowing and purring. We are interested in which pictures and sounds make your child smile. We will ask you (the parent) to turn around to avoid influencing your child's responses.", + purpose: "Why do babies love cats? This study will help us find out whether babies love cats because of their soft fur or their twitchy tails." + }; + + const hello_trial = { + type: jsPsychHtmlButtonResponse, + stimulus: 'Hello world!', + choices: ['Next'] + }; + + const exit_survey = { type: chsSurvey.ExitSurveyPlugin }; + + jsPsych.run([video_config, video_consent, hello_trial, exit_survey]); + + +Step 6: Add session recording +------------------------------------------------- + +We now have a bare-bones CHS experiment structure: webcam/mic configuration, video-recorded consent, our "experiment" (hello world trial), and an exit survey. The only thing we're missing for typical CHS experiments is webcam recordings that are captured during the experiment. We can do that using either: (a) session recording, where a webcam recording starts at a specified point in the experiment's trial sequence and ends at another point, with any number of experimental trials in between, or (b) trial recording, where a webcam recording is created at the start of a particular trial and ends when that trial finishes. + +.. admonition:: When should/shouldn't I use trial and session recording? + + CHS's trial and session recording features are intended for use with "normal" trials that do not already have any kind of webcam access/recording. Therefore you should NOT use trial or session recording during a ``VideoConfigPlugin`` or ``VideoConsentPlugin`` trial, and you should NOT use the trial recording and session recording features simultaneously for any trial(s). + + +We'll start by adding session recording that starts before our hello world trial and ends immediately afterwards. To do this, we need to make a few changes: + +1. Add a "start recording" trial using the ``StartRecordPlugin`` from the `CHS Record package `__. + + .. code:: javascript + + const startRec = { type: chsRecord.StartRecordPlugin }; + +2. Add a "stop recording" trial using the ``StopRecordPlugin`` from the `CHS Record package `__. + + .. code:: javascript + + const stopRec = { type: chsRecord.StopRecordPlugin }; + +3. Put these trials into the experiment timeline, before and after the "hello world" trial. + + .. code:: javascript + + jsPsych.run([video_config, video_consent, start, hello_trial, stop, exit_survey]); + +Here's the full experiment code now: + +.. code:: javascript + + const jsPsych = initJsPsych(); + + const video_config = { type: chsRecord.VideoConfigPlugin }; + + const video_consent = { + type: chsRecord.VideoConsentPlugin, + PIName: "Jane Smith", + institution: "Science University", + PIContact: "Jane Smith at 123 456 7890", + payment: "After you finish the study, we will email you a $5 BabyStore gift card within approximately three days. To be eligible for the gift card your child must be in the age range for this study, you need to submit a valid consent statement, and we need to see that there is a child with you. But we will send a gift card even if you do not finish the whole study or we are not able to use your child's data! There are no other direct benefits to you or your child from participating, but we hope you will enjoy the experience.", + procedures: "Your child will be shown pictures of lots of different cats, along with noises that cats make like meowing and purring. We are interested in which pictures and sounds make your child smile. We will ask you (the parent) to turn around to avoid influencing your child's responses.", + purpose: "Why do babies love cats? This study will help us find out whether babies love cats because of their soft fur or their twitchy tails." + }; + + const start = { type: chsRecord.StartRecordPlugin }; + + const hello_trial = { + type: jsPsychHtmlButtonResponse, + stimulus: 'Hello world!', + choices: ['Next'] + }; + + const stop = { type: chsRecord.StopRecordPlugin }; + + const exit_survey = { type: chsSurvey.ExitSurveyPlugin }; + + jsPsych.run([video_config, video_consent, start, hello_trial, stop, exit_survey]); + + +Step 7: Switch to trial recording +------------------------------------------------- + +We now we'll switch to trial-level recording during our hello world trial. To do this, we need to make a few changes: + +1. Remove the "start" and "stop" trials that we added in the last step. To do this, you can leave unused "start" and "stop" trials in your code and just remove them from the experiment timeline array passed to ``jsPsych.run``. Now your experiment timeline should look like this: + + .. code:: javascript + + jsPsych.run([video_config, video_consent, hello_trial, exit_survey]); + + +2. Add the `chsRecord.TrialRecordExtension` to the parameters that are passed into ``initJsPsych``. This is the `jsPsych initialization function `__, which takes an optional settings object and parameters. So far we haven't needed any of these setting parameters, but now we do need to tell jsPsych that we're planning to use the trial recording extension at some point during the experiment. + + .. code:: javascript + + const jsPsych = initJsPsych({ + extensions: [{ type: chsRecord.TrialRecordExtension }] + }); + + + +3. Add the `chsRecord.TrialRecordExtension` to the configuration for the trial that we want to be recorded. This tells jsPsych to run trial recording for that particular trial. Here we are just adding trial recording to the "hello_world" trial. + + .. code:: javascript + + const hello_trial = { + type: jsPsychHtmlButtonResponse, + stimulus: 'Hello world!', + choices: ['Next'], + extensions: [{ type: chsRecord.TrialRecordExtension }], // add this line + }; + +Here's what the whole experiment looks like now: + +.. code:: javascript + + const jsPsych = initJsPsych({ + extensions: [{ type: chsRecord.TrialRecordExtension }] + }); + + const video_config = { type: chsRecord.VideoConfigPlugin }; + + const video_consent = { + type: chsRecord.VideoConsentPlugin, + PIName: "Jane Smith", + institution: "Science University", + PIContact: "Jane Smith at 123 456 7890", + payment: "After you finish the study, we will email you a $5 BabyStore gift card within approximately three days. To be eligible for the gift card your child must be in the age range for this study, you need to submit a valid consent statement, and we need to see that there is a child with you. But we will send a gift card even if you do not finish the whole study or we are not able to use your child's data! There are no other direct benefits to you or your child from participating, but we hope you will enjoy the experience.", + procedures: "Your child will be shown pictures of lots of different cats, along with noises that cats make like meowing and purring. We are interested in which pictures and sounds make your child smile. We will ask you (the parent) to turn around to avoid influencing your child's responses.", + purpose: "Why do babies love cats? This study will help us find out whether babies love cats because of their soft fur or their twitchy tails." + }; + + const hello_trial = { + type: jsPsychHtmlButtonResponse, + stimulus: 'Hello world!', + choices: ['Next'], + extensions: [{ type: chsRecord.TrialRecordExtension }], + }; + + const exit_survey = { type: chsSurvey.ExitSurveyPlugin }; + + jsPsych.run([video_config, video_consent, hello_trial, exit_survey]); + + +Step 8: Learn more on the jsPsych website +------------------------------------------------- + +Well done! You've learned how to create a jsPsych experiment on CHS! You can use this tutorial code as a starting point for your real experiment, and replace the hello world trial with some actual jsPsych experiment code. + +Check out the jsPsych "hello world" tutorial if you want to start from the basics, or the `RT task tutorial `__ to learn the more advanced jsPsych features. + +There are also lots of examples on the jsPsych plugin documentation pages (e.g. `html-button-response plugin `__, `survey plugin `__), as well as in the `jsPsych repository "examples" folder `__. + +Keep in mind that our internal CHS jsPsych experiment runner offers a limited set of jsPsych packages (core library and plugins/extensions) and package versions. In the future, we plan to expand this set. Please see our :ref:`CHS jsPsych limitations page ` for information one what jsPsych packages are currently available. + + +Bonus: Translate your experiment into another language +------------------------------------------------------- + +Did you know that CHS offers translation options for several languages? By default, all of the "hard-coded" text in our custom CHS jsPsych plugins is presented in English, but you can change this using the optional ``locale`` parameter. This parameter is available in both the `CHS Record `__ plugins/extensions and the `CHS Surveys `__ plugins. + +Keep in mind that this ``locale`` parameter only affects the "hard-coded" text that is built in to our CHS plugins and extensions. It will not affect any of the text that you can already set with a parameter, e.g. the text that you provide for the video consent plugin. + +Also, all of the "official" jsPsych plugins/extensions already parameterize all of the text, which makes this translation parameter unnecessary. + +Try converting your tutorial experiment into another langauge! Here's what our tutorial code looks like with a French translation: + +.. code:: javascript + + const jsPsych = initJsPsych(); + + const video_config = { + type: chsRecord.VideoConfigPlugin, + locale: "fr" + }; + + const video_consent = { + type: chsRecord.VideoConsentPlugin, + PIName: "Jane Smith", + institution: "Science University", + PIContact: "Jane Smith at 123 456 7890", + payment: "After you finish the study, we will email you a $5 BabyStore gift card within approximately three days. To be eligible for the gift card your child must be in the age range for this study, you need to submit a valid consent statement, and we need to see that there is a child with you. But we will send a gift card even if you do not finish the whole study or we are not able to use your child's data! There are no other direct benefits to you or your child from participating, but we hope you will enjoy the experience.", + procedures: "Your child will be shown pictures of lots of different cats, along with noises that cats make like meowing and purring. We are interested in which pictures and sounds make your child smile. We will ask you (the parent) to turn around to avoid influencing your child's responses.", + purpose: "Why do babies love cats? This study will help us find out whether babies love cats because of their soft fur or their twitchy tails.", + locale: "fr" + }; + + const start = { + type: chsRecord.StartRecordPlugin, + locale: "fr" + }; + + const hello_trial = { + type: jsPsychHtmlButtonResponse, + stimulus: 'Hello world!', + choices: ['Next'] + }; + + const stop = { + type: chsRecord.StopRecordPlugin, + locale: "fr" + }; + + const exit_survey = { + type: chsSurvey.ExitSurveyPlugin, + locale: "fr" + }; + + jsPsych.run([video_config, video_consent, start, hello_trial, stop, exit_survey]); \ No newline at end of file From ee7fa3d3b52e4d2ed4f694a9a86f1156d98c2bd5 Mon Sep 17 00:00:00 2001 From: Becky Gilbert Date: Thu, 7 Nov 2024 11:42:58 -0800 Subject: [PATCH 04/24] add target label and update tutorial step number --- docs/source/tutorial-lookit/tutorial-first-study.rst | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/source/tutorial-lookit/tutorial-first-study.rst b/docs/source/tutorial-lookit/tutorial-first-study.rst index ead77813..c683958a 100644 --- a/docs/source/tutorial-lookit/tutorial-first-study.rst +++ b/docs/source/tutorial-lookit/tutorial-first-study.rst @@ -1,10 +1,12 @@ +.. _lookit-tutorial-first-study: + ######################################### 1. Setting up your first Lookit study ######################################### In this section, you will be creating your first Lookit study. You will learn how to find and use experiment components, specify your protocol, and test out and troubleshoot your study. -This section is designed for researchers planning to build an internal Lookit study. If your first study will be written with jsPsych, then you can see the :ref:`jspsych tutorial `. Or if your study involves a link to an experiment hosted somewhere else, skip to part 6 of this tutorial. +This section is designed for researchers planning to build an internal Lookit study. If your first study will be written with jsPsych, then you can see the :ref:`jspsych tutorial `. Or if your study involves a link to an experiment hosted somewhere else, skip to part 4 of this tutorial. Step 1: Clone the 'Lookit tutorial part 1' study ------------------------------------------------- From 33261501a0d982a7f5eb8a4bc0022642893367ce Mon Sep 17 00:00:00 2001 From: Becky Gilbert Date: Thu, 7 Nov 2024 11:43:52 -0800 Subject: [PATCH 05/24] add new tutorial step 3 create study section for nesting lookit and jspsych tutorials --- docs/source/tutorial-create-study.rst | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100644 docs/source/tutorial-create-study.rst diff --git a/docs/source/tutorial-create-study.rst b/docs/source/tutorial-create-study.rst new file mode 100644 index 00000000..3769c225 --- /dev/null +++ b/docs/source/tutorial-create-study.rst @@ -0,0 +1,12 @@ +3. Create a study +==================== + +At this point in the tutorial, you should decide whether you want your first CHS study to use the Lookit or jsPsych experiment runner. Then, just complete the tutorial for that section, and move on to :ref:`Step 4: Managing Study Data `. + +Or, if your study involves a link to an experiment or scheduling platform hosted somewhere else, you can skip to step 4 in the tutorial: :ref:`Managing Study Data `. + +.. toctree:: + :maxdepth: 2 + + Lookit + jsPsych \ No newline at end of file From 19fbb88aa7e46106917b0bc5e631f80cf3fe41ee Mon Sep 17 00:00:00 2001 From: Becky Gilbert Date: Thu, 7 Nov 2024 11:44:33 -0800 Subject: [PATCH 06/24] add target and update tutorial step number --- docs/source/tutorial-manage-data.rst | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/source/tutorial-manage-data.rst b/docs/source/tutorial-manage-data.rst index ee74a80e..5a63dc10 100644 --- a/docs/source/tutorial-manage-data.rst +++ b/docs/source/tutorial-manage-data.rst @@ -1,5 +1,7 @@ +.. _tutorial-manage-data: + ############################################# -6. Managing study data +4. Managing study data ############################################# Now that you have a good handle on how to set up the study protocol you want, it's time to look at how to manage your study: controlling who has access to what, starting and stopping data collection, reviewing and downloading your data, and contacting your participants. From c3816b65e38ae40f76b5cfaa1cc63a253c817c6e Mon Sep 17 00:00:00 2001 From: Becky Gilbert Date: Thu, 7 Nov 2024 11:44:57 -0800 Subject: [PATCH 07/24] update tutorial step number --- docs/source/tutorial-next-steps.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/tutorial-next-steps.rst b/docs/source/tutorial-next-steps.rst index acda5864..742e8f32 100644 --- a/docs/source/tutorial-next-steps.rst +++ b/docs/source/tutorial-next-steps.rst @@ -1,5 +1,5 @@ ################################## -7. Finishing steps +5. Finishing steps ################################## You're in the home stretch! Thanks for sticking with it. We'll wrap up by revisiting Github issues and briefly mentioning some of the advanced features you might use in the future. From 68b9d28d10dc207afce864f600740ee0e6ec8c1e Mon Sep 17 00:00:00 2001 From: Becky Gilbert Date: Thu, 7 Nov 2024 11:51:04 -0800 Subject: [PATCH 08/24] add new edit study design page to correspond with our now-split study details/design pages, move study design fields from study details page, add jsPsych info --- docs/source/researchers-edit-design.rst | 66 +++++++++++++++++++++++++ 1 file changed, 66 insertions(+) create mode 100644 docs/source/researchers-edit-design.rst diff --git a/docs/source/researchers-edit-design.rst b/docs/source/researchers-edit-design.rst new file mode 100644 index 00000000..58c2a50d --- /dev/null +++ b/docs/source/researchers-edit-design.rst @@ -0,0 +1,66 @@ +.. _study_design: + +################################## +Setting the study design +################################## + +After you've created a study of a specific type, you can set the value of some additional fields that are specific to that study type and define the behavior of your study. See each study type below to see the design fields for that type. + + +*************************** +Lookit (Ember Frameplayer) +*************************** + +Study protocol configuration +============================= + +This needs to be a valid JSON block describing the different frames (pages) of your study, and the sequence. This can be left blank at the time you initially create your study. For detailed information about specifying your study protocol, see `Building an Experiment`_. + +Experiment runner code URL and version +========================================== + +The "Experiment runner code URL" is a link to the `Ember Frame Player `_ code that will run your study. It's an ember app that can talk to our API. All the frames in the experiment are defined in Ember and there is an exp-player component that can cycle through these frames. You should leave this link as is, unless you have forked the Ember Frame Player Github repository (for instance, to create custom frame types) and want to use your fork instead. + +The "Experiment runner version" is a Github commit SHA that refers to a specific version of the Ember Frame Player that you want to use. Typically when you create a new study, you will want to leave this blank, which means that your study will use the latest version of the experiment runner. This field is available in case you need to roll back to a previous version, for instance if an update to the Ember Frame Player caused a problem for your experiment. + + +****************** +jsPsych +****************** + +jsPsych Experiment Code +============================= + +This is where you enter your jsPsych experiment code. This is the JavaScript code used to generate a jsPsych study, not the surrounding HTML. Please see our :ref:`CHS jsPsych documentation ` and :ref:`tutorial ` for more information about what jsPsych plugins and versions are automatically loaded for you to use, and how to load stimuli files. + +.. _`Building an Experiment`: researchers-create-experiment.html + + +****************** +External studies +****************** + +.. _study-url: + +Study URL +============================= + +The link that families should be redirected to when they click the "Participate now" button on a study detail page. For unscheduled/unmoderated studies, this will be the study itself (e.g. a Qualtrics survey). For moderated studies, it should be a link to a scheduling system (e.g. Calendly). + +When the family clicks the "Participate now" button for external studies, the link will automatically include two pieces of information as URL query parameters: the hashed child ID ('child') and the response ID ('response'). This will allow you to automatically capture and record this information on the study/scheduling page, so that you can link the study responses and child's CHS account without having to ask the family to enter addintional information. For example, if your Study URL is "\https://example.com", then the family will be directed to a link that has this format: + + \https://example.com/?child=SG7JLN&response=d5c8f502-6588-46c8-84fa-a9657a44fe47 + +It is up to the researcher to capture and record the "child" and "response" URL query parameter values on the external website. Many online experiment/survey tools have documentation on how to do this (e.g. `Qualtrics `_). You can include your own URL query parameters in your Study URL and they will be retained along with the CHS parameters. + +Scheduling Platform +============================= + +Choose from a set of options to help us understand how researchers schedule participants for moderated/scheduled studies, and to build tools for common study types. + +Study Platform +============================= + +Choose from a set of study platforms to help us understand & build tools for common study types. + + From 7d38884331a80ae50280ec9777f5d042a2e8e3d2 Mon Sep 17 00:00:00 2001 From: Becky Gilbert Date: Thu, 7 Nov 2024 11:56:48 -0800 Subject: [PATCH 09/24] update home page and TOC with CHS vs Lookit terms and to add jsPsych experiment runner --- docs/source/index.rst | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/docs/source/index.rst b/docs/source/index.rst index f1435263..ee07c7fb 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -1,23 +1,23 @@ -Welcome to Lookit's documentation! -========================================== +Welcome to the Children Helping Science documentation! +==================================================================== -Here you will find detailed information about how to use Lookit to conduct developmental research online, as well as how to contribute to the codebase. +Here you will find detailed information about how to use Children Helping Science (CHS) to conduct developmental research online, as well as how to contribute to the codebase. -If you're new to Lookit, check out the :ref:`Getting started ` guide. The best way to get up to speed on setting up your study is to work through the :ref:`tutorial`. +If you're new to CHS, check out the :ref:`Getting started ` guide. The best way to get up to speed on setting up your study is to work through the :ref:`tutorial`. -What is Lookit? +What is CHS? ---------------- -Lookit is an online platform for developmental research. Families can sign up or log in, provide some basic information about their children, and then take part in studies from a variety of labs when it's convenient for them. +CHS (formerly "Lookit") is an online platform for developmental research. Families can sign up or log in, provide some basic information about their children, and then take part in studies from a variety of labs when it's convenient for them. -There are current three main categories of study types on Lookit: +There are three main categories of study types on CHS: -1. Internal studies, using the Lookit experiment builder. The Lookit experiment builder can be used to collect looking measures from preverbal children as well as verbal responses, pointing, etc. from older children. These studies have no videoconferencing or scheduling: parents and children simply do activities in the web browser, and information about the session (e.g., survey responses, timing, condition assignment) along with webcam video of the session is sent to the Lookit server where it can be accessed by authorized researchers. +1. Internal studies, created using either the **Lookit** or **jsPsych** experiment builder. These experiment builders can run studies with webcam recordings, which means they can be used to collect looking measures from preverbal children as well as verbal responses, pointing, etc. from older children. These studies have no videoconferencing or scheduling: parents and children simply do activities in the web browser, and information about the session (e.g., survey responses, timing, condition assignment) along with webcam video of the session is sent to the CHS server where it can be accessed by authorized researchers. -2. External, asynchronous (unscheduled, unmoderated) studies. Reseachers provide a link to a study hosted elsewhere (e.g. Qualtrics), and families see these studies advertised together with the Lookit-internal studies. Demographic information about children who participate in a specific study is made available to the corresponding researchers, and many experiment platforms make it possible to store an anonymous session ID, in order to match Lookit data to the external study data. +2. External, asynchronous (unscheduled, unmoderated) studies. Reseachers provide a link to a study hosted elsewhere (e.g. Qualtrics), and families see these studies advertised together with the internal studies. Demographic information about children who participate in a specific study is made available to the corresponding researchers, and many experiment platforms make it possible to store an anonymous session ID, in order to match CHS data to the external study data. -3. External, sychronous (scheduled, moderated) studies. Researchers provide a link to a scheduling system, and arrange a time to meet a family for a session over video chat. Families can see which studies are scheduled vs. unscheduled, and researchers can access & link Lookit data in the same ways as for other external studies. +3. External, sychronous (scheduled, moderated) studies. Researchers provide a link to a scheduling system, and arrange a time to meet a family for a session over video chat. Families can see which studies are scheduled vs. unscheduled, and researchers can access & link CHS data in the same ways as for other external studies. -All study types use the Lookit platform to design, test, and manage studies, including study submission/posting, contacting participants and accessing participant data. For Lookit-internal studies, common tasks such as checking for informed verbal consent before accessing any data from a session are also built into the workflow. +All study types use the CHS platform to design, test, and manage studies, including study submission/posting, contacting participants and accessing participant data. For internal studies, common tasks such as checking for informed verbal consent before accessing any data from a session are also built into the workflow. Families may take part in studies from multiple labs over time. Having one central platform allows families to access many interesting studies for all the children in their family in one place, and researchers benefit from economies of scale in software development and recruitment. @@ -40,7 +40,7 @@ Families may take part in studies from multiple labs over time. Having one centr :maxdepth: 2 :hidden: :glob: - :caption: Using Lookit + :caption: Using CHS researchers-log-in researchers-manage-org @@ -51,7 +51,7 @@ Families may take part in studies from multiple labs over time. Having one centr :maxdepth: 2 :hidden: :glob: - :caption: Creating your study + :caption: Lookit studies researchers-set-study-fields researchers-create-experiment @@ -59,7 +59,7 @@ Families may take part in studies from multiple labs over time. Having one centr researchers-prep-stimuli researchers-style-guide researchers-lag-issues - Experiment runner releases + Releases .. toctree:: :maxdepth: 2 From 4bc05829b2639c5fc0c4c3b770468fbf3939f2b5 Mon Sep 17 00:00:00 2001 From: Becky Gilbert Date: Thu, 7 Nov 2024 11:58:40 -0800 Subject: [PATCH 10/24] update TOC with new create study step and nested lookit/jspsych tutorials --- docs/source/index.rst | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/docs/source/index.rst b/docs/source/index.rst index ee07c7fb..ff60b564 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -87,16 +87,15 @@ Families may take part in studies from multiple labs over time. Having one centr .. toctree:: - :maxdepth: 3 + :maxdepth: 4 :hidden: + :includehidden: :glob: :caption: Tutorial - tutorial-access + 1. Accessing CHS site and community tutorial-contributing - tutorial-first-study - tutorial-second-study - tutorial-protocol-exercises + tutorial-create-study tutorial-manage-data tutorial-next-steps From 230a9aceeeb1e91bff17d6808d680b7755ffbfa9 Mon Sep 17 00:00:00 2001 From: Becky Gilbert Date: Thu, 7 Nov 2024 12:00:36 -0800 Subject: [PATCH 11/24] move study fields page from Lookit into Using CHS since these apply to all study types, add new study design page --- docs/source/index.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/source/index.rst b/docs/source/index.rst index ff60b564..618b84a5 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -46,6 +46,8 @@ Families may take part in studies from multiple labs over time. Having one centr researchers-manage-org researchers-manage-studies researchers-study-permissions + researchers-set-study-fields + researchers-edit-design .. toctree:: :maxdepth: 2 @@ -53,7 +55,6 @@ Families may take part in studies from multiple labs over time. Having one centr :glob: :caption: Lookit studies - researchers-set-study-fields researchers-create-experiment researchers-update-code researchers-prep-stimuli From a59ea8bc00702ec0711f9dd8439f81ef147ea312 Mon Sep 17 00:00:00 2001 From: Becky Gilbert Date: Thu, 7 Nov 2024 12:04:57 -0800 Subject: [PATCH 12/24] remove fields that are specific to study type (now in on the study design page) and add study type section with jsPsych option --- docs/source/researchers-set-study-fields.rst | 47 +++----------------- 1 file changed, 6 insertions(+), 41 deletions(-) diff --git a/docs/source/researchers-set-study-fields.rst b/docs/source/researchers-set-study-fields.rst index 5e38c108..f81ab482 100644 --- a/docs/source/researchers-set-study-fields.rst +++ b/docs/source/researchers-set-study-fields.rst @@ -411,49 +411,14 @@ Language codes |yue |Yue | +----+----------------------+ - -========================================================= -Study protocol configuration (Lookit studies) -========================================================= -This needs to be a valid JSON block describing the different frames (pages) of your study, and the sequence. This can be left blank at the time you initially create your study. For detailed information about specifying your study protocol, see `Building an Experiment`_. - -========================================================= -Experiment runner code URL and version (Lookit studies) -========================================================= -The "Experiment runner code URL" is a link to the application you're using to enable participants to take a study. Right now, we just have one option, the `Ember Frame Player `_. It's an ember app that can talk to our API. All the frames in the experiment are defined in Ember and there is an exp-player component that can cycle through these frames. For details, see `Editing study type`_. You should leave this link as is, unless you have forked the Ember Frame Player Github repository (for instance, to create custom frame types) and want to use your fork instead. - -The "Experiment runner version" is a Github commit SHA that refers to a specific version of the Ember Frame Player version that you want to use. Typically you will want to leave this blank, which means that your study will use the latest version of the experiment runner. This field is available in case you need to roll back to a previous version, for instance if an update to the Ember Frame Player caused a problem for your experiment. - -.. _study-url: - ============================= -Study URL (External studies) +Experiment Type ============================= -The link that families should be redirected to when they click the "Participate now" button on a study detail page. For unscheduled/unmoderated studies, this will be the study itself (e.g. a Qualtrics survey). For moderated studies, it should be a link to a scheduling system (e.g. Calendly). - -When the family clicks the "Participate now" button for external studies, the link will automatically include two pieces of information as URL query parameters: the hashed child ID ('child') and the response ID ('response'). This will allow you to automatically capture and record this information on the study/scheduling page, so that you can link the study responses and child's CHS account without having to ask the family to enter addintional information. For example, if your Study URL is "\https://example.com", then the family will be directed to a link that has this format: - - \https://example.com/?child=SG7JLN&response=d5c8f502-6588-46c8-84fa-a9657a44fe47 - -It is up to the researcher to capture and record the URL query parameter values on the external website. Many online experiment/survey tools have documentation on how to do this (e.g. `Qualtrics `_). You can include your own URL query parameters in your Study URL and they will be retained along with the CHS parameters. - -=========================================== -Scheduling Platform (External studies) -=========================================== -Choose from a set of options to help us understand how researchers schedule participants for moderated/scheduled studies, and to build tools for common study types. - -=========================================== -Study Platform (External studies) -=========================================== -Choose from a set of study platforms to help us understand & build tools for common study types. - - -.. _`Building an Experiment`: researchers-create-experiment.html - -.. _`Experiment data`: researchers-experiment-data.html -.. _`Setup for custom frame development`: frame-dev-setup.html +Choose the type of experiment you are creating. This will change the fields that appear on the Study Details page that you see next, because there are certain fields that depend on the type of study that you're running. -.. _`Setting study parameters`: researchers-set-study-fields.html +- **Lookit**: This is CHS's custom experiment builder, also called the Ember Frameplayer (EFP). It allows you to build an experiment that is hosted on CHS using JSON or JavaScript. See the Lookit Ember Framplayer documentation for more information about what you can do with this experiment builder. +- **jsPsych**: This will allow you to build an experiment that is hosted on CHS using JavaScript and jsPsych. See the jsPsych documentation for more information about what you can do with this experiment builder, as well as the CHS jsPsych documentation for information about features that we've added such as webcam recording. +- **External**: Choose this option if you want to post a link to an external website. -.. _`Editing study type`: researchers-manage-studies.html#editing-study-type +.. _`Building an Experiment`: researchers-create-experiment.html \ No newline at end of file From 836a420f76836576a3b6bd448d3243e4dac2b5cc Mon Sep 17 00:00:00 2001 From: Becky Gilbert Date: Thu, 7 Nov 2024 12:11:03 -0800 Subject: [PATCH 13/24] update lookit-api installation page with jsPsych and EFP RecordRTC info --- docs/source/install-django-project.rst | 24 +++++++++++++++++++----- 1 file changed, 19 insertions(+), 5 deletions(-) diff --git a/docs/source/install-django-project.rst b/docs/source/install-django-project.rst index f5d503d6..d37658b6 100644 --- a/docs/source/install-django-project.rst +++ b/docs/source/install-django-project.rst @@ -1,13 +1,12 @@ Installation: lookit-api (Django project) ========================================= -``lookit-api`` is the codebase for Experimenter and Lookit, excluding the actual +``lookit-api`` is the codebase for everything on the CHS website, excluding the actual studies themselves. Any functionality you see as a researcher or a participant (e.g., signing up, adding a child, editing or deploying a study, downloading data) is part of the ``lookit-api`` repo. -This project is built using Django and PostgreSQL. (The studies -themselves use Ember.js; see Ember portion of codebase, -`ember-lookit-frameplayer `__.), +This project is built using Django and PostgreSQL. The internal studies themselves are written with front-end experiment building frameworks (see the Lookit Ember.js portion of codebase here: +`ember-lookit-frameplayer `__ and the jsPsych portion here: `lookit-jspsych `__). It was initially developed by the `Center for Open Science `__. If you install only the ``lookit-api`` project locally, you will be able @@ -150,7 +149,22 @@ the superuser you created earlier using manage.py. Handling video ~~~~~~~~~~~~~~ -This project includes an incoming webhook handler for an event generated +Both the jsPsych and Lookit experiment runners require AWS credentials for uploading video to AWS S3 buckets during experiments, and retrieving those video files through the website. This works slightly differently for the two experiment runners. + +For the Lookit experiment runner, the video uploading credentials are stored in the lookit-api .env file are AWS S3 region: ``S3_REGION``, bucket name: ``S3_BUCKET``, secret key ID: ``S3_ACCESS_KEY_ID``, and secret access key: ``S3_SECRET_ACCESS_KEY``. The IAM user/role associated with these credentials needs permission to access and upload objects into the corresponding bucket. + +For the jsPsych experiment runner, the .env file must contain the AWS S3 region: ``JSPSYCH_S3_REGION`` and bucket name: ``JSPSYCH_S3_BUCKET``. The secret credentials that are needed for uploading videos during the experiment are temporary and generated "on the fly" when a user starts the experiment. + +You will also need separate AWS credentials for that the site uses for retrieving videos and generating the jsPsych temporary uploading credentials. This information is stored in the .env file, and includes a secret key ID: ``AWS_ACCESS_KEY_ID`` and a secret key: ``AWS_SECRET_ACCESS_KEY``. The AWS IAM user/role associated with these credentials must have permission to access the two video storage buckets and retrieve objects - this allows the site to display consent videos and make the session recordings available to researchers (only if they have all of the required permissions!). This IAM user/role also needs permissions for generating the temporary credentials that are needed for jsPsych experiments, which includes permission to access and upload objects into the jsPsych bucket. + +After a video is uploaded to an AWS S3 bucket, it triggers an AWS Lambda function that notifies the website about the video. This is how information about videos is stored in our database. The Lambda function uses a secret key to create a signature that is added to the header of the video POST request, which the lookit-api must verify before accepting the POST request. Therefore the lookit-api requires the same secret key that is accessed by the Lambda function in order to generate the signature. This key is stored in the .env file as ``AWS_LAMBDA_SECRET_ACCESS_KEY``. + +Older versions of Lookit Studies +----------------------------------- + +Older versions of the Lookit experiment runner (prior to Jan 30 2024) upload videos through an out-of-network service called Pipe. If you are not worried about running studies that use this older experiment runner version, then you can skip this set up. + +The older Lookit experiment runner versions require an incoming webhook handler for an event generated by the Pipe video recording service used by ember-lookit-frameplayer when video is transferred to our S3 storage. This requires a webhook key for authentication. It can be generated via our Pipe account and, for local testing, stored in From a3ab316fabdc60aebe5008e715cc03060ea21f21 Mon Sep 17 00:00:00 2001 From: Becky Gilbert Date: Thu, 7 Nov 2024 12:41:55 -0800 Subject: [PATCH 14/24] update CHS vs Lookit terms, update URLs, split Study Edit section into Study Ad/Design, add Design options for jsPsych and external --- docs/source/researchers-manage-studies.rst | 105 +++++++++++++-------- 1 file changed, 64 insertions(+), 41 deletions(-) diff --git a/docs/source/researchers-manage-studies.rst b/docs/source/researchers-manage-studies.rst index 11d1047b..46d65012 100644 --- a/docs/source/researchers-manage-studies.rst +++ b/docs/source/researchers-manage-studies.rst @@ -2,34 +2,34 @@ Managing studies ################################## --------------------- +=================== Viewing study list --------------------- -To view all studies, navigate to /exp/studies/. (We will use this short format to indicate relative paths starting with the Lookit site you are using - e.g., https://lookit.mit.edu/exp/studies/ or https://lookit-staging.mit.edu/exp/studies/). From there, the researcher can only see studies they have permission to view. Lab members and admins can see all studies that belong to their lab. Otherwise, researchers can only view studies which they have created or to which they have been explicitly added. +=================== +To view all studies, navigate to /exp/studies/. (We will use this short format to indicate relative paths starting with the CHS site you are using - e.g., https://childrenhelpingscience.com/exp/studies/ or https://babieshelpingscience.com/exp/studies/). From there, the researcher can only see studies they have permission to view. Lab members and admins can see all studies that belong to their lab. Otherwise, researchers can only view studies which they have created or to which they have been explicitly added. You can filter studies by name or by keywords in the description. Additionally, you can sort on various study states like "Created" or "Submitted", or filter on your own studies by selecting "My Studies". You can also sort on study name, study end date, and study begin date. .. image:: _static/img/study_list.png :alt: Viewing studies --------------------- +=================== Creating a study --------------------- +=================== To create a study, click the green "Create Study" button on the study list page or navigate to /exp/studies/create/. You'll need to provide values for the fields as described in `Setting study fields`_. .. image:: _static/img/create_study.png :alt: Creating a study --------------------- +=================== Cloning a study --------------------- +=================== To clone a study, click the "Clone Study" button on an existing study detail page. In order to see this button, you will need to have permission to create studies in at least one lab - in general everyone has been added to the Sandbox lab, so even if you are not part of another lab, you will be able to clone studies you can see. Cloning creates a copy of the study but adds the logged in user as the creator and puts it in a lab where the user is able to create studies. No data is copied along with the study. The clone will be moved back into "Created" status (e.g., if the current study is actively collecting data, the cloned study will not be - it will need to be approved before it can be started). --------------------- +=================== Study detail page --------------------- +=================== To view a single study, click on it from the study list. A researcher must have permission to view this study specifically. Lab admins and lab members can view all studies in their lab. Other researchers can only view this study if they have been explicitly added to the study. At the top, you see many of the study details that you entered when you created the study. The UUID is also displayed; this is your study's unique identifier and is used in the direct link to the study. @@ -50,19 +50,19 @@ On this page, you can also view and edit study status, view and manage researche .. _study status: ----------------------------------------------------------------------- +============================================================================ Study status: submitting your study and managing data collection ----------------------------------------------------------------------- +============================================================================ .. image:: _static/img/study_status_cannot_change.png :alt: Study rejected status displayed without option to edit -On the study detail page, you can see the current status of a study. For users with permissions to change the study state there is also a dropdown with the available actions (start/pause data collection, submit for approval, etc.) as shown below. The available states where you can move the study depend on what state is next in the sequence, as well as your current level of permissions. For example, if a study's current state is "Created", that study can only be "Submitted" for review, or "Archived", which removes the study from display. Only Lookit admins can approve or reject a study. +On the study detail page, you can see the current status of a study. For users with permissions to change the study state there is also a dropdown with the available actions (start/pause data collection, submit for approval, etc.) as shown below. The available states where you can move the study depend on what state is next in the sequence, as well as your current level of permissions. For example, if a study's current state is "Created", that study can only be "Submitted" for review, or "Archived", which removes the study from display. Only CHS admins can approve or reject a study. .. image:: _static/img/study_status_can_change.png :alt: Study rejected status displayed without option to edit -New studies must be submitted and approved by Lookit before they can be started. The :ref:`study approval process ` is intended to give Lookit staff an opportunity to check that studies comply with the Terms of Use and to provide support if necessary. +New studies must be submitted and approved by CHS before they can be started. The :ref:`study approval process ` is intended to give CHS staff an opportunity to check that studies comply with the Terms of Use and to provide support if necessary. When you submit a study for approval, you will be asked to provide some information to help with the review process - indicating any changes since last approval if you are re-submitting a previously approved study, and indicating anything that requires special review if you are submitting for the first time. @@ -73,74 +73,79 @@ Researchers will receive email notifications when their study is approved or whe Once your study is approved, it is not automatically live and collecting data. Researchers with appropriate permissions can independently start/pause data collection at will; however, if any changes are made to the study it will be automatically rejected and will require re-approval. -Studies can only be submitted for approval once the associated Lab is approved to test on Lookit. If your Lab is not yet approved, for instance if you are using the Sandbox lab to try Lookit out, you will see a message like this: +Studies can only be submitted for approval once the associated Lab is approved to test on CHS. If your Lab is not yet approved, for instance if you are using the Sandbox lab to try CHS out, you will see a message like this: .. image:: _static/img/study_submit_not_until_approved_lab.png :alt: Study rejected status displayed without option to edit The Lab is approved to test once it has a signed access agreement and someone from the group has completed the terms of use quiz. This is update manually but generally within a day of completing those steps; please contact us if you think your lab is incorrectly not approved to test yet. -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Study state reference list -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +--------------------------- The possible study states are: :created: Study has been initially created, but has not been submitted for approval -:submitted: Study is submitted and awaiting approval by a Lookit admin -:approved: Study has been approved by a Lookit admin to run on Lookit, but is not yet active -:rejected: Changes have been requested by a Lookit admin before the study can be approved. The study should be edited before resubmitting. -:active: Study is active and collecting data; participants can access it at the study link. If the study is also marked "Discoverable", the study will show up on Lookit's study list. -:paused: Study is not actively collecting data or visible on Lookit; participants cannot access it. +:submitted: Study is submitted and awaiting approval by a CHS admin +:approved: Study has been approved by a CHS admin to run on CHS, but is not yet active +:rejected: Changes have been requested by a CHS admin before the study can be approved. The study should be edited before resubmitting. +:active: Study is active and collecting data; participants can access it at the study link. If the study is also marked "Discoverable", the study will show up on CHS's study list. +:paused: Study is not actively collecting data or visible on CHS; participants cannot access it. :deactivated: Study is done collecting data :archived: Study has been archived and removed from search ----------------------------------------------------------------------- +=============================================== Starting and stopping data collection ----------------------------------------------------------------------- +=============================================== Starting and pausing data collection can be done instantly at any time after your study is approved, using the same dropdown menu as for submitting your study. -What does "starting" your study do? If your study is set as "discoverable" (one of the checkboxes under "edit study"), starting will add your study to the set of studies displayed at ``_, and anyone (including you) will be able to participate in it from there. If your study is set as non-discoverable, anyone will be able to participate via a direct link (shown on your study page in the experimenter interface). This is useful for studies intended for a very specific population, for instance if you're doing an online follow-up to an in-person study: you can email the direct link to families, without worrying about screening out other families on Lookit. +What does "starting" your study do? If your study is set as "discoverable" (one of the checkboxes under "edit study"), starting will add your study to the set of studies displayed at ``_, and anyone (including you) will be able to participate in it from there. If your study is set as non-discoverable, anyone will be able to participate via a direct link (shown on your study page in the experimenter interface). This is useful for studies intended for a very specific population, for instance if you're doing an online follow-up to an in-person study: you can email the direct link to families, without worrying about screening out other families on CHS. --------------------- -Study edit page --------------------- -On the study edit page, you can update much of the metadata about the study. You can only view this page if you have permission to view the study details, meaning that you have been given a role specifically on this study OR you are a lab member. If you do not have permission to write study details, you will not be able to make any changes from this page. For more detail about the fields you can view and edit on this page, see "Setting study details." +=============================================== +Study ad page +=============================================== -To edit fields, change the information and click Save Changes in the middle of the page. If your study has already been approved, then the save button will be red. Otherwise it will be green. If your study has already been approved, then editing key details will automatically put the study in a rejected state. You must resubmit your -study and get it approved again by a Lookit admin to run the study on the Lookit platform. +On the study ad page, you can update much of the metadata about the study. You can only view this page if you have permission to view the study details, meaning that you have been given a role specifically on this study OR you are a lab member. If you do not have permission to write study details, you will not be able to make any changes from this page. For more detail about the fields you can view and edit on this page, see "Setting study details." -At the bottom of the edit study page, you can make edits to your study's structure (the frames, or pages, in your experiment), and the sequence of those frames. You can also make advanced edits to the commits we are using to build your study. +To edit fields, change the information and click Save Changes in the middle of the page. If your study has already been approved, then editing key details will automatically put the study in a rejected state. You must resubmit your +study and get it approved again by a CHS admin to run the study on the CHS platform. .. image:: _static/img/study_edit.png :alt: Editing studies +=============================================== +Study design page +=============================================== + +On the study design page, you can make edits to your study's structure. For internal Lookit and jsPsych studies, this will include things like: the frames/trials in your experiment, and the sequence of those frames/trials. With Lookit studies, you can also make advanced edits to the commits we are using to build your study. For external studies, this is where you can edit your study link. + + +Lookit studies -------------------------------------- -Editing study protocol configuration --------------------------------------- + +Protocol configuration +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + For more information about how to specify what happens during your study, see `Building an Experiment`_. The study protocol configuration specifies the frames (or pages) of your experiment, and also specifies the order they go in. -To edit a study's protocol, click 'Edit study' from the study detail page. You will only be able to make actual changes from this page if you are a lab admin, or have a study admin, design, or manager role. +To edit a study's protocol, click 'Edit Study Design' from the study detail page. You will only be able to make actual changes from this page if you are a lab admin, or have a study admin, design, or manager role. Click on the JSON block. A JSON editor will appear. Click on "Beautify" in the top right corner for better readability. Note that any invalid JSON will be shown via a little red X at the left of the relevant line! Once you are happy with your changes click 'Close'. Then hit "Save Changes" in the bottom right corner. -If your study has already been approved, then clicking "Save Changes" will automatically reject the study. You will have to resubmit it for a Lookit admin to reapprove. +If your study has already been approved, then clicking "Save Changes" will automatically reject the study. You will have to resubmit it for a CHS admin to reapprove. .. image:: _static/img/json_editor.png :alt: Edit JSON To preview your study, click "See Preview". (You will need to build an experiment runner first if you haven't yet, or if you've changed the version you're using.) --------------------------------- -Editing experiment runner type --------------------------------- -To edit the type of experiment runner used by your study, click 'Edit study' from the study detail page and scroll down to the bottom of the page. -The experiment runner is the application you're using to enable participants to take a study. Right now, we just have one option, the `Ember Frame Player `_. It's an ember app that can talk to our API. All the frames in the experiment are defined in ember-lookit-frameplayer, and the exp-player component can cycle through these frames. +Experiment runner options +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -**If you don't want any customization and want to use the existing player and frames, just select the defaults.** These are advanced options! +**If you don't want any customization and want to use the most up-to-date version of the Lookit Ember Frameplayer and frames, just select the defaults.** These are advanced options! What does each field mean? @@ -150,6 +155,24 @@ What does each field mean? **Important:** Whenever you update the code versions you are using, you will need to re-build your re-build your experiment runner before you can preview or run your study. This build process creates your very own runner application using exactly the code you selected, so that your study will continue to run as you designed it. You only need to re-build these when you have changed the code URL or version - not when you update your study protocol configuration or other data like the age range. + +jsPsych studies +---------------------------------- + +For jsPsych studies, the "Edit Study Design" page allows you to edit the JavaScript code that defines your experiment. See the :ref:`jsPsych tutorial ` for a demonstration of how to use the "jsPsych Experiment Code" box. + +Unlike the Lookit Ember Frameplayer experiment runner, with jsPsych studies there is no "experiment build" step, and there are no configuration options for selecting different jsPsych packages/versions. After adding/editing your jsPsych experiment code in this JavaScript editor, you can run through your study immediately. + + +External studies +---------------------------------- + +For external studies, the "Edit Study Design" page allows you to edit the following details about your study: + +- Whether your study is scheduled (i.e. one-on-one appointments with a researcher) or unmoderated (i.e. participants use a link to take a study on their own). +- The study URL. This is where the participant will be directed when they click "Participate now!" or "Schedule a time to participate". +- Information about the study scheduling and/or hosting platform that you're using. + .. _`Building an Experiment`: researchers-create-experiment.html .. _`Experiment data`: researchers-experiment-data.html From 0ac5df089c45e461411485da1393eabfc6a25a1c Mon Sep 17 00:00:00 2001 From: Becky Gilbert Date: Thu, 7 Nov 2024 13:10:46 -0800 Subject: [PATCH 15/24] update CHS vs Lookit terms, CHS URLs, add jsPsych study type and link to tutorial --- docs/source/researchers-start-here.rst | 51 +++++++++++--------------- 1 file changed, 21 insertions(+), 30 deletions(-) diff --git a/docs/source/researchers-start-here.rst b/docs/source/researchers-start-here.rst index f6b80f10..54824195 100644 --- a/docs/source/researchers-start-here.rst +++ b/docs/source/researchers-start-here.rst @@ -17,11 +17,13 @@ the new improved Children Helping Science platform, here's how you get started! - Bring-your-own Meeting: You provide a link to a scheduling page for families (e.g. Calendly). A researcher meets the family on a video call to conduct the session. - - Lookit Experiment: You build a study in the Lookit experiment builder, which is - customized for developmental studies and has :ref:`features ` + - Build it here: You build an 'internal' study using one of our experiment builders, which have been + customized for developmental studies and have :ref:`features ` including webcam recording. Families take the study on their own time, without a researcher present. + * Lookit: This is our original experiment runner software that was custom built for developmental studies. + * jsPsych: This uses our `customized version `_ of the `jsPsych `_ experiment building software. Join the Slack workspace @@ -34,7 +36,7 @@ Slack is our primary means of communication about new features, best practices, etc. It’s also where you should go with any questions or technical support requests, and for peer review from other researchers. There are a lot of participating researchers -(over 800 at last count!) who may be able to help! +(over 2500 at last count!) who may be able to help! If you also want to receive updates via email, please join the `lookit-research `__ list as well. @@ -43,9 +45,9 @@ If you also want to receive updates via email, please join the Overview - 15 minute to your first study! ------------------------------------------- -1. Check to see if we already have an agreement with your institution to use the online testing platform (~1 minute) -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Approximately 100 institutions have already signed agreements. +1. Check to see if we already have an agreement with your institution (~1 minute) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +An institutional agreement is required to use our online testing platform. Over 100 institutions have already signed agreements. a) Check whether your institution is listed `here `__. @@ -55,18 +57,17 @@ If your institution is not listed, the form you need is `here `__ to make a new researcher account. +Go `here `__ to make a new researcher account. Please don't use a single login for your whole lab - even if you use a single email for lab communications (like mydevelopmentallab@gmail.com). Sharing login information presents a security risk for participants' data. -Each researcher using the platform should `create their own account `__. +Each researcher using the platform should `create their own account `__. 3. Create (~5 minutes) or join (~1 minute) your lab account ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -If you are the first user from your lab, go `here `__ to -create your own “lab” on the platform. +If you are the first user from your lab, go `here `__ to create your own “lab” on the platform. Visit `this link `__ to review and take a short quiz on our terms of use. One person from your lab needs to @@ -74,17 +75,14 @@ complete this quiz before your lab is approved to submit studies. You will use your lab to manage access to your group's studies. Every lab has its own public page that shows the studies that are active right now. (Example: -https://lookit.mit.edu/studies/early-childhood-cognition-lab/) +https://childrenhelpingscience.com/studies/early-childhood-cognition-lab/) If your lab already has an account, go -to `this page `__, find your lab and click -"Request to join". +to `this page `__, find your lab and click "Request to join". -4. Create a study (~10 minutes if you already have a study set up that you want to post) +4. Create a study (~10 minutes if you already have a study ready to post) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -To create a study, go `here `__ and check -the “External” checkbox if you are posting a link to an existing study you have -already set up (bring-your-own study link or bring-your-own meeting.) +If you are posting a link to an existing study you have already set up (bring-your-own study link or bring-your-own meeting), go `here `__ to create a new study, and select the “External” option under "Experiment Type". Any studies you run will need to be approved by your institution's IRB (just like studies you run in person). If you do not already have permission to conduct @@ -98,14 +96,11 @@ IRB of the amendment. .. admonition:: Human subjects approval on the platform - The Lookit platform is run through MIT, but MIT's IRB is not involved in any - studies from other institutions (unless an MIT researcher is your actual research - collaborator on a study!) + The Children Helping Science platform is run through MIT, but MIT's IRB is not involved in any studies from other institutions (unless an MIT researcher is your actual research collaborator on a study!) If you do not yet have a study design and want to learn how to use the platform to create studies (including asynchronous studies with no experimenter present that will collect webcam -data of your participants) visit the :ref:`Lookit tutorial ` to learn how -the platform works. +data of your participants) visit the :ref:`Lookit tutorial ` or the :ref:`jsPsych tutorial `. 5. What happens next? ^^^^^^^^^^^^^^^^^^^^^^^ @@ -146,10 +141,7 @@ is in the specified age range, will only pay once per child) or to avoid paying who do not participate in good faith (see below). However, payment needs to be for participants' time and effort - without prior approval -by Lookit, payment may *not* depend on the child's behavior or performance, even if -that behavior renders data unusable (e.g., if an infant fusses and his parent ends -the session early). Similarly, payment may not depend on the parent's video privacy -selections. +by CHS, payment may *not* depend on the child's behavior or performance, even if that behavior renders data unusable (e.g., if an infant fusses and his parent ends the session early). Similarly, payment may not depend on the parent's video privacy selections. Beginning December 2023, new studies are required to confirm that study participants are acting in good faith (i.e. that a child of the correct age is present and the family is @@ -162,7 +154,7 @@ You can learn more about this process and about preventing spam participation on Yes. You can ensure that any families you recruit go directly to your study page on the website by including a direct link to that study anywhere you advertise -for participants. `Here `__ is an example of a direct link. +for participants. `Here `__ is an example of a direct link. **Do some studies have priority over others on the platform?** @@ -188,7 +180,7 @@ part of your lab or personal researcher website. This might look like this: "Did you know that you can participate in research from the comfort of your own home? `Click here to check out Children Helping Science, an online platform -for developmental research. `__ Families can +for developmental research. `__ Families can participate in our studies on their home computers, any time they want!" @@ -198,8 +190,7 @@ Recommendations for your first online study If you are in the process of planning your first online study, here is an ordering of steps we recommend that you take: -1. For internal studies, complete the :ref:`Lookit tutorial ` to get familiar with how the platform - works and how to implement a study on it. +1. For internal studies, complete the :ref:`tutorial ` to get familiar with how the platform works and how to implement a study on it. During this tutorial, you can choose to create your first study using either the :ref:`Lookit EFP ` or :ref:`jsPsych ` experiment runner. 2. Figure out the details of how your study will work - counterbalancing, practice trials, etc. Draft the parent-facing instructions, record any audio and video From 2496344044cb40d685cc331a1772b471b35f713a Mon Sep 17 00:00:00 2001 From: Becky Gilbert Date: Thu, 7 Nov 2024 13:19:32 -0800 Subject: [PATCH 16/24] update CHS vs Lookit terms, CHS links, add jsPsych experiment runner, update subprocessor info --- .../community-irb-and-legal-information.rst | 78 +++++++++---------- 1 file changed, 38 insertions(+), 40 deletions(-) diff --git a/docs/source/community-irb-and-legal-information.rst b/docs/source/community-irb-and-legal-information.rst index 45d93668..f5ec2d4e 100644 --- a/docs/source/community-irb-and-legal-information.rst +++ b/docs/source/community-irb-and-legal-information.rst @@ -6,13 +6,13 @@ IRB and legal info Overview --------- -Before you can run a study on Lookit, there are three things you'll need to do: +Before you can run a study on CHS, there are three things you'll need to do: 1) Sign and return the access agreement (details below). This covers your entire lab. You only have to do this once. 2) Get IRB approval at your own institution (details below). -3) :ref:`Get peer feedback on the study, and then have it approved by Lookit. ` +3) :ref:`Get peer feedback on the study, and then have it approved by CHS. ` Access agreement ---------------------- @@ -20,18 +20,18 @@ Access agreement What it is ~~~~~~~~~~ -In order to use Lookit for data collection, the principal investigator (PI) of a research group must first sign +In order to use CHS for data collection, the principal investigator (PI) of a research group must first sign an `institutional agreement `__ with MIT that lays out each institution’s rights and responsibilities. This may require review by a legal or contracts office or an office of sponsored programs depending on the institution. -The access agreement with a PI at a given institution covers **all** studies conducted on Lookit where the approved IRB protocol at that institution names that person as the PI. (If that institution cedes to another institution for approval, that's still okay.) +The access agreement with a PI at a given institution covers **all** studies conducted on CHS where the approved IRB protocol at that institution names that person as the PI. (If that institution cedes to another institution for approval, that's still okay.) The agreement itself is short and closely based on Databrary’s agreement (with permission); the meat is in the `Terms of -Use `__ you are agreeing to. +Use `__ you are agreeing to. **Researchers should actually read the terms carefully:** these are real content written by a real person and contains non-obvious information about ethical/privacy issues you may not have encountered in the lab. Note @@ -39,31 +39,31 @@ that we do impose standards that may in some cases be more stringent than your own IRB’s. To make sure we’re all on the same page, we require that at least one -researcher associated with each research group using Lookit complete `this quiz +researcher associated with each research group using CHS complete `this quiz about the Terms of Use `__. How to get it signed ~~~~~~~~~~~~~~~~~~~~ -If you are planning a Lookit study, getting the access agreement signed +If you are planning a CHS study, getting the access agreement signed should be your first step because it takes a highly variable amount of time (record so far is under two weeks, but it can take a few months). The access agreement doesn't require any customization on your part - you just need to send it off to someone at your institution who’s authorized to sign on behalf of the institution. -First, check whether your institution already has an access agreement set up. You can check the list `here `__ or email lookit@mit.edu. If so, you'll be able to just add yourself (newer form of agreement) or get in touch with the person who signed off previously (older form of agreement). If there were any modifications to the standard agreement made for your institution, you'll be able to just use the modified form instead of having to re-negotiate the same changes. +First, check whether your institution already has an access agreement set up. You can check the list `here `__ or email childrenhelpingscience@gmail.com. If so, you'll be able to just add yourself (newer form of agreement) or get in touch with the person who signed off previously (older form of agreement). If there were any modifications to the standard agreement made for your institution, you'll be able to just use the modified form instead of having to re-negotiate the same changes. If there isn't an existing agreement, you'll need to figure out which office to send it to for approval. Unfortunately the appropriate office varies by institution. If you’ve previously set up to use Databrary, try contacting the office that signed off on that. As examples, the following offices have approved the access agreement: Contracts, VP Finance, Partnerships, University/Industry Liason, Department head, Provost, Sponsored programs. -In general, you do NOT need to have an IRB protocol to use Lookit before the access agreement can be signed. In some cases, though, the office that signs the agreement may ask that you obtain ethical approval first. +In general, you do NOT need to have an IRB protocol to use CHS before the access agreement can be signed. In some cases, though, the office that signs the agreement may ask that you obtain ethical approval first. The one exception to this process is that researchers at MIT, completing studies approved by MIT’s IRB, do not require the signature of an MIT authorized signer. However, the MIT PI still needs to sign and return the agreement. -Correspondence between access agreements and Lookit Labs +Correspondence between access agreements and CHS Labs ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -In general, each Authorized Investigator on an access agreement should correspond to exactly one Lab on Lookit, which makes it far more straightforward to ensure that everyone's actually covered. +In general, each Authorized Investigator on an access agreement should correspond to exactly one Lab on CHS, which makes it far more straightforward to ensure that everyone's actually covered. -Each Lab must be associated with at least one agreement. (E.g., suppose Jill is a PI at Science University and has signed an access agreement, and Joe is another PI at Science University who has not. We cannot approve Joe's Lab on Lookit to test - even if he tells us that his work will always actually be covered under Jill's IRB protocol.) +Each Lab must be associated with at least one agreement. (E.g., suppose Jill is a PI at Science University and has signed an access agreement, and Joe is another PI at Science University who has not. We cannot approve Joe's Lab on CHS to test - even if he tells us that his work will always actually be covered under Jill's IRB protocol.) However, if multiple PIs at your institution are closely affiliated (e.g. a lab jointly headed by two professors, or students all serve as PIs on IRB protocols in your department) please go ahead and create a lab listing multiple PIs and just let us know. We'd need an access agreement for each PI, and would expect all studies conducted by this lab to be covered by an IRB protocol listing at least one of those PIs. @@ -80,43 +80,43 @@ institution, please get in touch with CHS staff (childrenhelpingscience@gmail.co counsel lawyer Jason Baletsa (jbaletsa at mit.edu) for advice. Please note that approval of changes happens on the MIT General Counsel office's -timeline, which is not under Lookit's control. +timeline, which is not under CHS's control. -Sending it to Lookit +Sending it to CHS ~~~~~~~~~~~~~~~~~~~~ Once you have the user agreement signed by the PI and institution’s authorized signer, please: - Make sure you or someone in your group has completed `this quick quiz about the Terms of Use `__ -- Send the PDF of the signed agreement to lookit@mit.edu, noting that you have also completed the quiz +- Send the PDF of the signed agreement to childrenhelpingscience@gmail.com, noting that you have also completed the quiz .. _irb: -Ethics approval for Lookit studies +Ethics approval for CHS studies ------------------------------------ -A researcher using Lookit applies to their own institution’s IRB for +A researcher using CHS applies to their own institution’s IRB for approval. Except for our involvement in a small set of initial studies, MIT is not considered to be engaged in human subjects research by -running Lookit. +running CHS. You can find resources for writing your IRB protocol -`here `__. You will likely need to give a brief overview of how data collection on Lookit works and what protections are included for participant data. +`here `__. You will likely need to give a brief overview of how data collection on CHS works and what protections are included for participant data. -Lookit *may* request a copy of the approved protocol at the time a study -is submitted to run on Lookit. (In general we don’t - we aren’t looking +CHS *may* request a copy of the approved protocol at the time a study +is submitted to run on CHS. (In general we don’t - we aren’t looking to get into that much paperwork. This is mostly if we have any concerns or you’re getting permission to do something unusual.) How consent and assent work ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The consent process on Lookit is fairly standardized by design, to reduce the mental load on families. Parents consent by recording a verbal statement of consent after reviewing a document. +The consent process on CHS is fairly standardized by design, to reduce the mental load on families. Parents consent by recording a verbal statement of consent after reviewing a document. -Information about what the consent page looks like to parents, and how to customize it, is available :ref:`here ` in the frame documentation. You can see the available templates for the consent document `here `__. Essentially you will supply pieces of text that will be substituted into the template. +Information about what the consent page looks like to parents, and how to customize it, is available in the Lookit EFP documentation :ref:`here ` and in the CHS jsPsych documentation `here `__. You can see the available templates for the consent document `here `__. Essentially you will supply pieces of text that will be substituted into the template. -If you are testing older children and need to collect assent in addition, you will use the +If you are testing older children and need to collect assent in addition, you will use the Lookit :ref:`assent page `, which is less standardized at this point. When applying for IRB approval, you may need to present the exact text of the consent document that will be presented, which you can do by substituting values into the templates linked above. You can even start setting up your study (copy one of the tutorial/example studies and customize the consent text) and preview it to get screenshots or the PDF download. @@ -128,7 +128,7 @@ Please keep the text you insert as easy to read as you possibly can. Do not incl Researchers must use these standard consent pages, barring extraordinary legal barriers at their institutions (we have not encountered anyone not able to approve a protocol). You may NOT use your existing consent form instead or in addition just because you don't want to submit an amendment to your protocol (sorry). You will likely need to apply for a waiver of the usual requirement to collect written -consent from participants so that you can rely on videorecorded statements. (The statements parents make on Lookit are better evidence of true informed consent than commonly-approved checkboxes for online studies, and so far this has not been an issue.) +consent from participants so that you can rely on videorecorded statements. (The statements parents make during Lookit studies are better evidence of true informed consent than commonly-approved checkboxes for online studies, and so far this has not been an issue.) You can see an example `here `__. Responsibilities of researchers @@ -136,20 +136,20 @@ Responsibilities of researchers - Everyone: - - Protect your Lookit account credentials. **You are responsible for - any access to participant data on Lookit via your account.** You + - Protect your CHS account credentials. **You are responsible for + any access to participant data on CHS via your account.** You should choose a strong password, change it regularly, and never share it. (Share access by adding another researcher to your Lab - not by sharing a lab account password.) - Promptly report any breach or potential breach of participant - information that you become aware of, both to Lookit and to your + information that you become aware of, both to CHS and to your local institution. - PIs: - You are responsible for the actions of anyone who’s conducting - research on Lookit under your access agreement. Ensure that anyone - added to your Lab on Lookit understands any rules in the Terms of + research on CHS under your access agreement. Ensure that anyone + added to your Lab on CHS understands any rules in the Terms of Use that apply to what they’ll be doing. (E.g., make sure an RA tasked with approving consent and contacting families with compensation understands when it’s ok to withhold compensation, @@ -164,7 +164,7 @@ Responsibilities of researchers - Study admins: - Ensure you have current IRB approval for your study before - beginning ANY data collection on Lookit. Promptly pause the study + beginning ANY data collection on CHS. Promptly pause the study if approval lapses. - Ensure that anyone with access to participant data for this study is listed on the appropriate IRB protocol. Promptly remove @@ -176,19 +176,17 @@ Privacy policy ----------------- Both researchers and participants are covered by the `privacy -policy `__. +policy `__. Sub-processors and information about GDPR compliance/DPAs ----------------------------------------------------------- -AWS S3 (data storage - participant video): -https://aws.amazon.com/service-terms/ +- AWS S3 (data storage - participant video): https://aws.amazon.com/service-terms/ -GCP (data storage, databases; temporary data storage, video (.zip -downloads); platform and study hosting/deployment): -https://cloud.google.com/security/gdpr/resource-center/contracts-and-terms +- GCP (data storage, databases; temporary data storage, video .zip downloads; platform and study hosting/deployment): https://cloud.google.com/security/gdpr/resource-center/contracts-and-terms -Pipe (video streaming; no copies stored): https://addpipe.com/gdpr +- Sentry (error reporting; no identifiable information): https://sentry.io/security/#hipaa-and-hitech -Sentry (error reporting; no identifiable information): -https://sentry.io/security/#hipaa-and-hitech +Only applies to studies using an older Lookit experiment runner (versions released before Jan 30 2024): + +- Pipe (video streaming; no copies stored): https://addpipe.com/gdpr From 9a498040997a3d55145c6f0fc9b96c3aac7241e2 Mon Sep 17 00:00:00 2001 From: Becky Gilbert Date: Thu, 7 Nov 2024 14:40:01 -0800 Subject: [PATCH 17/24] update with CHS vs Lookit terms, and add jsPsych to architecture overview --- .../contribute-architecture-overview.rst | 62 ++++++++++--------- 1 file changed, 33 insertions(+), 29 deletions(-) diff --git a/docs/source/contribute-architecture-overview.rst b/docs/source/contribute-architecture-overview.rst index d8aa230e..7e914d24 100644 --- a/docs/source/contribute-architecture-overview.rst +++ b/docs/source/contribute-architecture-overview.rst @@ -1,20 +1,24 @@ ================================== -Overview of Lookit architecture +Overview of CHS architecture ================================== Codebase and stack ------------------- -Although we present participants and researchers with an integrated experience that has an intuitive user interface, behind-the-scenes the Lookit codebase is three independent projects: +Although we present participants and researchers with an integrated experience that has an intuitive user interface, behind-the-scenes the CHS codebase is four independent projects: -- lookit-api is the primary application that supports study management and participation. It contains a Django (Python) application, which provides an HTTP/JSON API consumed by built experiments as well as web UIs for researchers to design and manage experiments, for participants to discover experiments, and for administrators to manage accounts and static website content. It also contains offline Celery processes (connected by a RabbitMQ message queue) to build and deploy experiments using Docker, and to send email. Django provides a solid framework and abstractions for development of views for both participants and experimenters. -- ember-lookit-frameplayer is the “experiment runner” that parses a JSON study protocol and presents a sequence of interactive pages to the participant. It is implemented in the Javascript framework Ember, which is well-suited to the structure of providing a “library” of reusable, customizable frame components. New components are written in Javascript, HTML (Handlebars), and SCSS. Parameters and outputs of these components are automatically documented using Yuidoc, along with example screenshots and videos. This experiment runner is entirely separable from the other elements of Lookit; the code is already well structured to accommodate alternate experiment types. -- lookit-orchestrator contains the continuous integration and continuous deployment (CI/CD) tools for Lookit - in particular, tools for automatically testing, building, and deploying code as changes are pushed to GitHub. This encourages the frequent incorporation of small changes to make development more robust. We use a Kustomize-based workflow with a standard containerized GitOps execution pathway. +- `lookit-api `__ is the primary application that supports CHS website, including study management and participation. It contains a Django (Python) application, which provides an HTTP/JSON API consumed by built experiments as well as web UIs for researchers to design and manage experiments, for participants to discover experiments, and for administrators to manage accounts and static website content. It also contains offline Celery processes (connected by a RabbitMQ message queue) to build and deploy experiments using Docker, and to send email. Django provides a solid framework and abstractions for development of views for both participants and experimenters. +- `ember-lookit-frameplayer `__ is the internal Lookit experiment runner. It parses a JSON study protocol and presents a sequence of interactive pages to the participant. It is implemented in the Javascript framework Ember, which is well-suited to the structure of providing a “library” of reusable, customizable frame components. New components are written in Javascript, HTML (Handlebars), and SCSS. Parameters and outputs of these components are automatically documented using Yuidoc, along with example screenshots and videos. This experiment runner is entirely separable from the other elements of CHS; the code is already well structured to accommodate alternate experiment types. +- `lookit-jspsych `__ is the internal jsPsych experiment runner. It uses our set of custom jsPsych packages that (1) handle the API calls for updating response data in the lookit-api database, and (2) allow the experiment to record webcam video and upload the files to AWS S3. This repository also contains the jsPsych versions of standard CHS trial types, including: video configuration, video consent, and an exit survey. The lookit-jsPsych packages exist as an add-on to "normal" jsPsych experiments - researchers mostly write their experiments using JavaScript and the jsPsych library, and the lookit-jsPsych packages exist to provide these additional features. Like the Ember Lookit Frameplayer, the jsPsych experiment runner is generally separate from the rest of CHS, with the exception that lookit-jsPsych experiments run inside a lookit-api template (rather than proxying to a different hosting location, in the case of Lookit EFP). The lookit-jspsych packages are written in Typescript, HTML (Handlebars) and SCSS. We also use Rollup for bundling and the Jest framework for tests. The documentation comes from the package READMEs using Read the Docs and MkDocs. +- `lookit-orchestrator `__ contains the continuous integration and continuous deployment (CI/CD) tools for CHS - in particular, tools for automatically testing, building, and deploying code as changes are pushed to GitHub. This encourages the frequent incorporation of small changes to make development more robust. We use a Kustomize-based workflow with a standard containerized GitOps execution pathway. -There are three major out-of-network service dependencies: +There are two major out-of-network service dependencies: + +- Amazon S3/Lambda for primary storage of participant video +- Google Cloud Storage for storing Lookit experiment web archives and other static files + +And an additional dependency that is only used in older versions of the Lookit experiment runner (released prior to January 30 2024). -- Amazon S3 for primary storage of participant video, -- Google Cloud Storage for storing experiment web archives and other static files, and - Pipe to relay video streamed from experiments to Amazon S3 upon verification. We also use Sendgrid for sending email to researchers and participants from the lookit-api web UI, and Sentry for frontend and backend error logging. @@ -22,46 +26,46 @@ We also use Sendgrid for sending email to researchers and participants from the How it all works together -------------------------- -The Lookit ecosystem can be described as a sort of "restaurant" architecture, if the restaurant had a prix fixe menu. It's a cheeky and imperfect analogy, but it gets the general idea across. In this analogy there are a couple of main players you need to know about: +The CHS ecosystem can be described as a sort of "restaurant" architecture, if the restaurant had a prix fixe menu. It's a cheeky and imperfect analogy, but it gets the general idea across. In this analogy there are a couple of main players you need to know about: 1. **experiments** are *meals* - - packaged/built web archives loaded into Google Cloud Storage (GCS) and proxied/served by the web application (see point 4 below) - - Right now, the only experiment type we support is `ember-lookit-frameplayer `__; theoretically we could create adapters for lookit for different experiment front-ends. + - Lookit experiments are packaged/built web archives loaded into Google Cloud Storage (GCS) and proxied/served by the web application (see point 4 below). + - jsPsych experiments are served directly by the CHS website (lookit-api) through a page that loads all of the lookit-jspsych dependencies and the researcher's experiment code. Unlike Lookit studies, these are not pre-built or stored on GCS. 2. **researchers** are *head chefs* - - they design experiments and then build their dependencies with a process that uploads them to GCS. + - They design experiments using either a JSON protocol structure or JavaScript protocol generator (Lookit studies), or using JavaScript (jsPsych). + - For Lookit studies, researchers also build their dependencies with a process that uploads the experiment files to GCS. 3. **participants** are *customers* - - they use the wait staff to order meals that are on the menu. + - They use the wait staff to order meals that are on the menu. 4. The **web application** is your *wait staff* - - They show off the menu to customers - - They also take orders from the head chef to add, remove, or change items on the menu, and relay those to the kitchen (see point 5 below) + - They show off the menu to customers. + - They also take orders from the head chef to add, remove, or change items on the menu, and relay those to the kitchen (see point 5 below). 5. The **builder** is your *kitchen* - - receives orders from the head chef by way of the wait staff. - - actually cooks "meals" (builds experiments) + - It receives orders from the head chef by way of the wait staff. + - It actually cooks "meals" (builds experiments, in the case Lookit studies). - RabbitMQ kind of acts like a stack of order slips, in this system. 6. The **worker** is kind of like a *general manager* - - sends out mail (email) to previous customers - - occasionally, you'll see him sweeping the kitchen (cleaning up junk left over from docker builds) - -The "Frameplayer" and video data -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -`Ember-lookit-frameplayer `_ is an Ember application that can be "built" into a web archive (WAR) with bundled/minimized Javascript and CSS. It's the default experiment type, and currently the only one available. + - It sends out mail (email) to previous customers. + - Occasionally, you'll see him sweeping the kitchen (cleaning up junk left over from docker builds). -To capture video data, we use `Pipe `_. The JS client is basically a plugin that is parameterized during the WAR build process; when properly configured it will stream video data to Pipe's servers. This data is then uploaded to Amazon S3, which, upon completion, triggers a webhook that fires a POST payload to a special ` handler in our API `_ that finds the video in S3 and renames it to something more searchable. +The experiment runners and video data +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +We currently support two options for experiment runners. `Ember-lookit-frameplayer `_ is an Ember application that can be "built" into a web archive (WAR) with bundled/minimized Javascript and CSS. The `lookit-jsPsych `__ experiment runner allows users to write an experiment using JavaScript and the jsPsych library, which runs as a single-page web app directly in a lookit-api view. +To capture video data, both the Ember-lookit-frameplayer (recent versions) and the jsPsych experiment runner upload video data directly from the client to AWS S3 storage. This is done by providing AWS upload credentials to the client during the experiment and managing the upload using the AWS JavaScript SDK. The creation of a new video object in S3 triggers an AWS Lambda function, which POSTs video metadata to the lookit-api and adds an entry for that video in our database. +Older versions of the Ember Lookit Frameplayer (released prior to Jan 30 2024) use `Pipe `_ to capture and stream video data. The JS client is basically a plugin that is parameterized during the WAR build process; when properly configured it streams video data to Pipe's servers. This data is then uploaded to Amazon S3, which, upon completion, triggers a webhook that fires a POST payload to a special `handler in our API `_ that finds the video in S3 and renames it to something more searchable. Cluster layout ------------------------ @@ -97,18 +101,18 @@ Lookit is organized as a collection of Kubernetes services, backed by deployment - ``-google-storage`` is basically just an external service that we set up to allow nginx ingress to reroute requests for static assets to GCS. -Setup for a separate instance of Lookit +Setup for a separate instance of CHS ---------------------------------------- Please contact us if you are looking to run your own instance; it will be a good idea to work together closely to both get you up and running, which will also provide critical feedback in making this pipeline more adaptable. -A good place to start if you are interested in running your own separate instance of Lookit is the `lookit-orchestrator README `__. To fill in a bit more: +A good place to start if you are interested in running your own separate instance of CHS is the `lookit-orchestrator README `__. To fill in a bit more: Google Cloud Platform (GCP) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ We rely almost exclusively on GCP components to orchestrate the app. A Cloud Builder Github integration trigger is tripped on deployments to either the "develop" or "master" branches of ``lookit-api``, executing the "CI" piece of the pipeline (`testing and containerization `_). You can see in the ``deploy-to-cluster`` step that the "CD" (`deployment `_) piece is executed near the very end. It leverages the contents of *this* repo, which are similarly containerized (using the GitHub integration for build triggers) and loaded into GCR for use as a `custom builder `_. -The CI pipeline is not completely generalized/parameterized, so to run your own Lookit CI pipeline, you'll want to set up your own brand new environment apart from the one that is used by the MIT instance of Lookit. To accomplish this, you'll need to set up your own Google Cloud Platform project. You'll need a few things turned on: +The CI pipeline is not completely generalized/parameterized, so to run your own CHS CI pipeline, you'll want to set up your own brand new environment apart from the one that is used by the MIT instance of CHS. To accomplish this, you'll need to set up your own Google Cloud Platform project. You'll need a few things turned on: - Kubernetes Engine - Cloud Builder @@ -117,7 +121,7 @@ The CI pipeline is not completely generalized/parameterized, so to run your own Once those services are turned on, you'll want to turn your focus to the GKE setup that is tuned by the lookit-orchestrator repo. -Kubernetes (`lookit-orchestrator` configuration) +Kubernetes (``lookit-orchestrator`` configuration) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ So far, we see quite a number of "in-network" players (webapp, builder, worker, etc.) and "out-of-network" services (Pipe, S3, Google Cloud Storage, Google Cloud SQL). While Kustomize and Kubernetes work tightly together to connect "in-network" players, "out-of-network" services all need login credentials, which are safely and securely built From 7062e35377200586b0537e4341a5bd5b92676423 Mon Sep 17 00:00:00 2001 From: Becky Gilbert Date: Thu, 7 Nov 2024 15:19:16 -0800 Subject: [PATCH 18/24] fix typo --- docs/source/tutorial-access.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/tutorial-access.rst b/docs/source/tutorial-access.rst index dc2afd4a..f79c1251 100644 --- a/docs/source/tutorial-access.rst +++ b/docs/source/tutorial-access.rst @@ -7,7 +7,7 @@ Welcome ---------- -Welcome to the Children Helping Scienc (CHS) tutorial! In this tutorial you will work through a series of exercises intended to build all the skills you'll need to run your own Lookit study on the CHS platform. By the end of the tutorial, you will be able to create a participant-friendly study that includes a survey and test trials, assign children to different experimental conditions, test and troubleshoot your study, download study data, contact participants, use and even edit this documentation! +Welcome to the Children Helping Science (CHS) tutorial! In this tutorial you will work through a series of exercises intended to build all the skills you'll need to run your own Lookit study on the CHS platform. By the end of the tutorial, you will be able to create a participant-friendly study that includes a survey and test trials, assign children to different experimental conditions, test and troubleshoot your study, download study data, contact participants, use and even edit this documentation! This tutorial was written for researchers who will be building a Lookit study, but most steps are also applicable for researchers who will be building a jsPsych study or sharing a link to an external study instead. From 213b00490c4e97ce21aadcff6c4fabd52942b418 Mon Sep 17 00:00:00 2001 From: Melissa Kline Struhl Date: Fri, 8 Nov 2024 10:18:04 -0500 Subject: [PATCH 19/24] Update README.md subtle changes to Lookit/CHS nomenclature --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 31e1eab7..8513b025 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # lookit-docs -Documentation for Children Helping Science (formerly "Lookit") project, including use of the CHS platform, the internal Lookit and jsPsych experiment runners, and associated repositories. +Documentation for Children Helping Science (aka "Lookit") project, including use of the CHS platform, the internal Lookit and jsPsych experiment runners, and associated repositories. CHS platform: https://lookit.readthedocs.io/en/develop/ @@ -306,4 +306,4 @@ Below are references for embedded/reusable links and footnotes (see https://gith ``` / $ git clone https://github.com//lookit-docs.git / $ cd lookit-docs - ``` \ No newline at end of file + ``` From add782039ef7085eab0200d6a11a840a174841d4 Mon Sep 17 00:00:00 2001 From: Melissa Kline Struhl Date: Fri, 8 Nov 2024 10:22:44 -0500 Subject: [PATCH 20/24] Update community-irb-and-legal-information.rst slight language update --- docs/source/community-irb-and-legal-information.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/source/community-irb-and-legal-information.rst b/docs/source/community-irb-and-legal-information.rst index f5ec2d4e..fd108c3e 100644 --- a/docs/source/community-irb-and-legal-information.rst +++ b/docs/source/community-irb-and-legal-information.rst @@ -6,9 +6,9 @@ IRB and legal info Overview --------- -Before you can run a study on CHS, there are three things you'll need to do: +Before you can run a study on CHS/Lookit, there are three things you'll need to do: -1) Sign and return the access agreement (details below). This covers your entire lab. You only have to do this once. +1) Sign and return the Lookit access agreement (details below). This covers your entire lab. You only have to do this once. 2) Get IRB approval at your own institution (details below). @@ -20,7 +20,7 @@ Access agreement What it is ~~~~~~~~~~ -In order to use CHS for data collection, the principal investigator (PI) of a research group must first sign +In order to use CHS/Lookit for data collection (including posting studies that run on other platforms/video chat studies), the principal investigator (PI) of a research group must first sign an `institutional agreement `__ with MIT that lays out each institution’s rights and responsibilities. From a2e4a8cb9c90c58737af5f6780f699b5abf13ec4 Mon Sep 17 00:00:00 2001 From: Melissa Kline Struhl Date: Fri, 8 Nov 2024 10:36:08 -0500 Subject: [PATCH 21/24] Update faq.rst Updating some nomenclature information - Becky, can you please check for any link display issues? --- docs/source/faq.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/source/faq.rst b/docs/source/faq.rst index abf0d0bb..2e335a39 100644 --- a/docs/source/faq.rst +++ b/docs/source/faq.rst @@ -4,9 +4,10 @@ FAQ What is the name of this platform? Children Helping Science or Lookit? --------------------------------------------------------------------------- -Children Helping Science (CHS) is the name of the website, and Lookit is the name of the 'default' experiment runner. +Children Helping Science (CHS) is the name of the website, and is the primary name that participants see when they are recruited to studies. The whole platform was called Lookit when it was first created, but in May 2023, it merged with a separate-but-similar project called Children Helping Science. Now, the platform (website) is called Children Helping Science (CHS), and the 'default' internal experiment runner is called Lookit. + +The Lookit name is also still used in other places, for instance, the names of our open-source code repositories - the codebase for the CHS website is "lookit-api" and the CHS documentation is "lookit-docs". Old links and email addresses are also still functioning, so don't worry about backwards compatibility: links like lookit.mit.edu/whatever - forward to childrenhelpingscience.com/whatever. Lookit is also still the official name of the project for things like the access agreements and some grants. -The whole platform was called Lookit when it was first created. In May 2023, it merged with a separate-but-similar project called Children Helping Science. Now, the platform (website) is called Children Helping Science (CHS), and the internal experiment runner is called Lookit. So if you're used to using the "Lookit" name for the whole site, you'll still see that term used to refer to a specific study type, i.e. studies that use the ember-lookit-frameplayer. You'll also see some remnants of the previous naming system in, for instance, the names of our open-source code repositories - the codebase for the CHS website is "lookit-api" and the CHS documentation is "lookit-docs". What is the name of the internal experiment runner? LoOkIt, Look!t, LooKit? -------------------------------------------------------------------------------------- From 416f2de29138cd0d84294b0f6517040f47236014 Mon Sep 17 00:00:00 2001 From: Becky Gilbert Date: Fri, 8 Nov 2024 10:07:39 -0800 Subject: [PATCH 22/24] fix typos and grammar errors --- docs/source/install-django-project.rst | 6 +++--- docs/source/researchers-edit-design.rst | 4 ++-- .../tutorial-jspsych/tutorial-jspsych-first-study.rst | 4 ++-- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/source/install-django-project.rst b/docs/source/install-django-project.rst index d37658b6..6ed47b08 100644 --- a/docs/source/install-django-project.rst +++ b/docs/source/install-django-project.rst @@ -149,13 +149,13 @@ the superuser you created earlier using manage.py. Handling video ~~~~~~~~~~~~~~ -Both the jsPsych and Lookit experiment runners require AWS credentials for uploading video to AWS S3 buckets during experiments, and retrieving those video files through the website. This works slightly differently for the two experiment runners. +Both the jsPsych and Lookit experiment runners require AWS credentials for uploading video to AWS S3 buckets during experiments and retrieving those video files through the website. This works slightly differently for the two experiment runners. -For the Lookit experiment runner, the video uploading credentials are stored in the lookit-api .env file are AWS S3 region: ``S3_REGION``, bucket name: ``S3_BUCKET``, secret key ID: ``S3_ACCESS_KEY_ID``, and secret access key: ``S3_SECRET_ACCESS_KEY``. The IAM user/role associated with these credentials needs permission to access and upload objects into the corresponding bucket. +For the Lookit experiment runner, the video uploading credentials stored in the lookit-api .env file are AWS S3 region: ``S3_REGION``, bucket name: ``S3_BUCKET``, secret key ID: ``S3_ACCESS_KEY_ID``, and secret access key: ``S3_SECRET_ACCESS_KEY``. The IAM user/role associated with these credentials needs permission to access and upload objects into the corresponding bucket. For the jsPsych experiment runner, the .env file must contain the AWS S3 region: ``JSPSYCH_S3_REGION`` and bucket name: ``JSPSYCH_S3_BUCKET``. The secret credentials that are needed for uploading videos during the experiment are temporary and generated "on the fly" when a user starts the experiment. -You will also need separate AWS credentials for that the site uses for retrieving videos and generating the jsPsych temporary uploading credentials. This information is stored in the .env file, and includes a secret key ID: ``AWS_ACCESS_KEY_ID`` and a secret key: ``AWS_SECRET_ACCESS_KEY``. The AWS IAM user/role associated with these credentials must have permission to access the two video storage buckets and retrieve objects - this allows the site to display consent videos and make the session recordings available to researchers (only if they have all of the required permissions!). This IAM user/role also needs permissions for generating the temporary credentials that are needed for jsPsych experiments, which includes permission to access and upload objects into the jsPsych bucket. +You will also need separate AWS credentials that the site uses for retrieving videos and generating the jsPsych temporary uploading credentials. This information is stored in the .env file, and includes a secret key ID: ``AWS_ACCESS_KEY_ID`` and a secret key: ``AWS_SECRET_ACCESS_KEY``. The AWS IAM user/role associated with these credentials must have permission to access the two video storage buckets and retrieve objects - this allows the site to display consent videos and make the session recordings available to researchers (only if they have all of the required permissions!). This IAM user/role also needs permissions for generating the temporary credentials that are needed for jsPsych experiments, which includes permission to access and upload objects into the jsPsych bucket. After a video is uploaded to an AWS S3 bucket, it triggers an AWS Lambda function that notifies the website about the video. This is how information about videos is stored in our database. The Lambda function uses a secret key to create a signature that is added to the header of the video POST request, which the lookit-api must verify before accepting the POST request. Therefore the lookit-api requires the same secret key that is accessed by the Lambda function in order to generate the signature. This key is stored in the .env file as ``AWS_LAMBDA_SECRET_ACCESS_KEY``. diff --git a/docs/source/researchers-edit-design.rst b/docs/source/researchers-edit-design.rst index 58c2a50d..51a7169f 100644 --- a/docs/source/researchers-edit-design.rst +++ b/docs/source/researchers-edit-design.rst @@ -19,7 +19,7 @@ This needs to be a valid JSON block describing the different frames (pages) of y Experiment runner code URL and version ========================================== -The "Experiment runner code URL" is a link to the `Ember Frame Player `_ code that will run your study. It's an ember app that can talk to our API. All the frames in the experiment are defined in Ember and there is an exp-player component that can cycle through these frames. You should leave this link as is, unless you have forked the Ember Frame Player Github repository (for instance, to create custom frame types) and want to use your fork instead. +The "Experiment runner code URL" is a link to the `Ember Frame Player `_ code that will run your study. It's an ember app that can talk to our API. All the frames in the experiment are defined in Ember, and there is an exp-player component that can cycle through these frames. You should leave this link as is, unless you have forked the Ember Frame Player Github repository (for instance, to create custom frame types) and want to use your fork instead. The "Experiment runner version" is a Github commit SHA that refers to a specific version of the Ember Frame Player that you want to use. Typically when you create a new study, you will want to leave this blank, which means that your study will use the latest version of the experiment runner. This field is available in case you need to roll back to a previous version, for instance if an update to the Ember Frame Player caused a problem for your experiment. @@ -47,7 +47,7 @@ Study URL The link that families should be redirected to when they click the "Participate now" button on a study detail page. For unscheduled/unmoderated studies, this will be the study itself (e.g. a Qualtrics survey). For moderated studies, it should be a link to a scheduling system (e.g. Calendly). -When the family clicks the "Participate now" button for external studies, the link will automatically include two pieces of information as URL query parameters: the hashed child ID ('child') and the response ID ('response'). This will allow you to automatically capture and record this information on the study/scheduling page, so that you can link the study responses and child's CHS account without having to ask the family to enter addintional information. For example, if your Study URL is "\https://example.com", then the family will be directed to a link that has this format: +When the family clicks the "Participate now" button for external studies, the link will automatically include two pieces of information as URL query parameters: the hashed child ID ('child') and the response ID ('response'). This will allow you to automatically capture and record this information on the study/scheduling page, so that you can link the study responses and child's CHS account without having to ask the family to enter additional information. For example, if your Study URL is "\https://example.com", then the family will be directed to a link that has this format: \https://example.com/?child=SG7JLN&response=d5c8f502-6588-46c8-84fa-a9657a44fe47 diff --git a/docs/source/tutorial-jspsych/tutorial-jspsych-first-study.rst b/docs/source/tutorial-jspsych/tutorial-jspsych-first-study.rst index ace4629a..bc7a8737 100644 --- a/docs/source/tutorial-jspsych/tutorial-jspsych-first-study.rst +++ b/docs/source/tutorial-jspsych/tutorial-jspsych-first-study.rst @@ -333,7 +333,7 @@ Check out the jsPsych "hello world" tutorial if you want to start from the basic There are also lots of examples on the jsPsych plugin documentation pages (e.g. `html-button-response plugin `__, `survey plugin `__), as well as in the `jsPsych repository "examples" folder `__. -Keep in mind that our internal CHS jsPsych experiment runner offers a limited set of jsPsych packages (core library and plugins/extensions) and package versions. In the future, we plan to expand this set. Please see our :ref:`CHS jsPsych limitations page ` for information one what jsPsych packages are currently available. +Keep in mind that our internal CHS jsPsych experiment runner offers a limited set of jsPsych packages (core library and plugins/extensions) and package versions. In the future, we plan to expand this set. Please see our :ref:`CHS jsPsych limitations page ` for information on what jsPsych packages are currently available. Bonus: Translate your experiment into another language @@ -345,7 +345,7 @@ Keep in mind that this ``locale`` parameter only affects the "hard-coded" text t Also, all of the "official" jsPsych plugins/extensions already parameterize all of the text, which makes this translation parameter unnecessary. -Try converting your tutorial experiment into another langauge! Here's what our tutorial code looks like with a French translation: +Try converting your tutorial experiment into another language! Here's what our tutorial code looks like with a French translation: .. code:: javascript From 0b45686f1510a6f08747c360fcb7473f8b766a3b Mon Sep 17 00:00:00 2001 From: Becky Gilbert Date: Fri, 8 Nov 2024 14:09:16 -0800 Subject: [PATCH 23/24] add jspsych studies section with intro, limitations and releases pages --- docs/source/index.rst | 10 +++ docs/source/researchers-jspsych-intro.rst | 68 +++++++++++++++++++ .../researchers-jspsych-limitations.rst | 20 ++++++ docs/source/researchers-jspsych-releases.rst | 7 ++ 4 files changed, 105 insertions(+) create mode 100644 docs/source/researchers-jspsych-intro.rst create mode 100644 docs/source/researchers-jspsych-limitations.rst create mode 100644 docs/source/researchers-jspsych-releases.rst diff --git a/docs/source/index.rst b/docs/source/index.rst index 618b84a5..1f748379 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -62,6 +62,16 @@ Families may take part in studies from multiple labs over time. Having one centr researchers-lag-issues Releases +.. toctree:: + :maxdepth: 2 + :hidden: + :glob: + :caption: jsPsych studies + + researchers-jspsych-intro + Limitations + Releases + .. toctree:: :maxdepth: 2 :hidden: diff --git a/docs/source/researchers-jspsych-intro.rst b/docs/source/researchers-jspsych-intro.rst new file mode 100644 index 00000000..0ebab68e --- /dev/null +++ b/docs/source/researchers-jspsych-intro.rst @@ -0,0 +1,68 @@ +.. _jspsych-intro: + +==================================== +jsPsych studies on CHS +==================================== + +`jsPsych `__ is an open source library for creating a wide range of behavioral experiments that run in a web browser. It consists of a core library and "plugins". Plugins are the components that make up the experiment. Each plugin lets you define different kinds of events (e.g. presenting an image or text) and it collects different kinds of data (e.g. responses and response times). The jsPsych library also supports a wide range of features that are commonly used in behavioral experiments, like looping over sets of trials, randomizing, sampling, and conditional behavior. + +To get a feel for how to build a jsPsych experiment, you can check out the jsPsych `Hello World tutorial `__ and `RT task tutorial `__. We also have our own :ref:`CHS jsPsych Hello World tutorial `, which demonstrates the CHS-specific features (:ref:`detailed in this section `) that we've added for jsPsych studies running on the CHS platform. + +To learn more about jsPsych features, some other great places to start are the jsPsych documentation about `timelines `__ and `dynamic parameters `__. + + +.. _jspsych-packages: + +jsPsych packages +============================== + +The CHS jsPsych experiment runner automatically loads a set of packages from the standard jsPsych library for researchers to use. These are: + +- `Core jsPsych library `__ v8.0.3 +- `HTML keyboard response plugin `__ v2.0.0: ``jsPsychHtmlKeyboardResponse`` +- `Image keyboard response plugin `__ v2.0.0: ``jsPsychImageKeyboardResponse`` +- `HTML button response plugin `__ v2.0.0: ``jsPsychHtmlButtonResponse`` +- `Image button response plugin `__ v2.0.0: ``jsPsychImageButtonResponse`` +- `Preload plugin `__ v2.0.0: ``jsPsychPreload`` + +We will likely add more options in the future. If there are any specific jsPsych plugins/extensions that your experiment needs, please let us know! The best way to request access to a standard jsPsych package is by creating a ``lookit-api`` `Github issue `__, but you can also let us know on Slack. + + +.. _chs-jspsych-packages: + +Custom CHS jsPsych packages +================================================== + +In addition to the jsPsych packages listed above, the CHS jsPsych experiment runner also automatically loads some custom packages. These custom plugins/extensions were designed to "fill in the gaps" in the sort of functionality that CHS researchers typically need for child development studies. This functionality includes: standardized webcam/mic configuration steps, video-recorded consent, trial/session recording, and standardized exit surveys. + +The `CHS jsPsych documentation `__ contains more information about all of the parameters available in the CHS jsPsych plugins/extensions listed below. + +Do you need any types of trials (Lookit EFP "frames") or features that are not listed here, and are not available through the standard jsPsych library? Let us know! The best way to request a custom (CHS-specific) jsPsych plugin/extension or feature is by creating a ``lookit-jspsych`` `Github issue `__, but you can also let us know on Slack. + +.. _chs-jspsych-plugins: + +Plugins +--------------------- + +- `Video config plugin `__ in the CHS Record package: ``chsRecord.VideoConfigPlugin`` +- `Video consent plugin `__ in the CHS Record package: ``chsRecord.VideoConsentPlugin`` +- `Start session recording plugin `__ in the CHS Record package: ``chsRecord.StartRecordPlugin`` +- `Stop session recording plugin `__ in the CHS Record package: ``chsRecord.StopRecordPlugin`` +- `Consent survey plugin `__ in the CHS Surveys package: ``chsSurveys.ConsentSurveyPlugin`` +- `Exit survey plugin `__ in the CHS Surveys package: ``chsSurveys.ExitSurveyPlugin`` + +.. _chs-jspsych-extensions: + +Extensions +-------------------------- + +jsPsych extensions are packages that extend the functionality of a plugin. + +- `Trial recording extension `__ in the CHS Record package: ``chsRecord.TrialRecordExtension`` + +.. _chs-jspsych-translations: + +Translations +-------------------------- + +All of these plugins/extensions support the automatic translation of hard-coded text through a ``locale`` parameter. diff --git a/docs/source/researchers-jspsych-limitations.rst b/docs/source/researchers-jspsych-limitations.rst new file mode 100644 index 00000000..bb5905c1 --- /dev/null +++ b/docs/source/researchers-jspsych-limitations.rst @@ -0,0 +1,20 @@ +.. _jspsych-limitations: + +==================================== +Limitations to CHS jsPsych studies +==================================== + +While building your jsPsych study on the CHS platform, you may run into problems because of certain limitations to the CHS jsPsych experiment runner. This is especially likely if you're an experienced jsPsych user with experiments that you want to move onto CHS, or if you're referencing other (non-CHS) jsPsych tutorials/examples to build your study on CHS. + +Here are some things you should keep in mind when creating jsPsych studies on CHS. + +- **All of the code needed to run your experiment must be provided in the CHS JavaScript editor.** + * Unlike many other jsPsych hosting options, on CHS you do not write an HTML file. CHS provides your study's HTML file and loads the JavaScript and CSS files. You can think of the CHS JavaScript editor as the main ``script`` element that you'll see in most jsPsych tutorials/examples. + * You do not have the option to load additional files, including stimuli files or custom plugins/extensions. + * Your stimuli files must be hosted online and loaded from an external URL, for instance via institutional storage, cloud storage, or Github (see our documentation about :ref:`hosting your stimuli online `). + +- **The CHS jsPsych experiment runner supports a limited set of jsPsych package/versions.** + * Please refer to our list of available :ref:`jsPsych packages/versions `, and our list of :ref:`custom CHS jsPsych packages `. + * If you're trying to use a jsPsych plugin that is not available on CHS, you will see the following console error when you try to run the study: "Uncaught ReferenceError: is not defined". + * If your jsPsych study uses plugins that are not available on CHS, you may be able to replace them with plugins that we do support. For instance, instructions that are presented with the ``instructions`` plugin could be presented via HTML keyboard/button response trials. + * If you have existing study code that uses a different version of the jsPsych core library or plugins, you may be able to update/modify your jsPsych code so that it's compatible with the versions we support. The jsPsych documentation offers version migration guides in their "Support" section, and all jsPsych packages contain release notes with detailed explanations of changes. \ No newline at end of file diff --git a/docs/source/researchers-jspsych-releases.rst b/docs/source/researchers-jspsych-releases.rst new file mode 100644 index 00000000..d4aa6233 --- /dev/null +++ b/docs/source/researchers-jspsych-releases.rst @@ -0,0 +1,7 @@ +.. _jspsych-releases: + +==================================== +jsPsych experiment runner updates +==================================== + +No updates yet. Watch this space! \ No newline at end of file From f043b55858eba37be1a93bfa223718565d8fa83f Mon Sep 17 00:00:00 2001 From: Becky Gilbert Date: Fri, 8 Nov 2024 14:24:03 -0800 Subject: [PATCH 24/24] add links to new jsPsych studies pages --- docs/source/researchers-edit-design.rst | 2 +- docs/source/tutorial-jspsych/tutorial-jspsych-first-study.rst | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/researchers-edit-design.rst b/docs/source/researchers-edit-design.rst index 1c4e07c0..51a7169f 100644 --- a/docs/source/researchers-edit-design.rst +++ b/docs/source/researchers-edit-design.rst @@ -31,7 +31,7 @@ jsPsych jsPsych Experiment Code ============================= -This is where you enter your jsPsych experiment code. This is the JavaScript code used to generate a jsPsych study, not the surrounding HTML. Please see our CHS jsPsych documentation and :ref:`tutorial ` for more information about what jsPsych plugins and versions are automatically loaded for you to use, and how to load stimuli files. +This is where you enter your jsPsych experiment code. This is the JavaScript code used to generate a jsPsych study, not the surrounding HTML. Please see our :ref:`CHS jsPsych documentation ` and :ref:`tutorial ` for more information about what jsPsych plugins and versions are automatically loaded for you to use, and how to load stimuli files. .. _`Building an Experiment`: researchers-create-experiment.html diff --git a/docs/source/tutorial-jspsych/tutorial-jspsych-first-study.rst b/docs/source/tutorial-jspsych/tutorial-jspsych-first-study.rst index a8056588..bc7a8737 100644 --- a/docs/source/tutorial-jspsych/tutorial-jspsych-first-study.rst +++ b/docs/source/tutorial-jspsych/tutorial-jspsych-first-study.rst @@ -333,7 +333,7 @@ Check out the jsPsych "hello world" tutorial if you want to start from the basic There are also lots of examples on the jsPsych plugin documentation pages (e.g. `html-button-response plugin `__, `survey plugin `__), as well as in the `jsPsych repository "examples" folder `__. -Keep in mind that our internal CHS jsPsych experiment runner offers a limited set of jsPsych packages (core library and plugins/extensions) and package versions. In the future, we plan to expand this set. Please see our CHS jsPsych limitations page for information on what jsPsych packages are currently available. +Keep in mind that our internal CHS jsPsych experiment runner offers a limited set of jsPsych packages (core library and plugins/extensions) and package versions. In the future, we plan to expand this set. Please see our :ref:`CHS jsPsych limitations page ` for information on what jsPsych packages are currently available. Bonus: Translate your experiment into another language