resources.md

layout	title
page	Resources

We’ve collected some links that we think will be helpful, but they are more like overviews than resources that will help you with a specific thing you are trying to accomplish.

Mechanical turk

In this class we will be running our replication projects on Amazon's Mechanical Turk (AMT/mTurk). mTurk is a platform on which Workers (or "Turkers") can complete Human Intelligence Tasks (HITs) for monetary compensation. HITs are put up by Requesters. mTurk originally was set up to do large-scale tasks that require human intelligence (e.g. labeling photos, finding telephone numbers, etc), but has recently been used by social scientists to conduct (large-scale) online experiments.

All replication projects will be launched using a common class account. This saves you the hassle of applying for a personal Requester account (which lately has become not-as-straightforward...), and also simplifies funding logistics. We will be giving more detailed instructions in class on how to access the class account and launch HITs etc.

A note on design considerations. Some Turkers multitask, and most of them do HITs for many hours a day, so appropriate designs would include attention checks, manipulation checks, etc.

A note on ethics. Note that Amazon workerIDs are actually tied to their (public) Amazon account, and thus constitute identifiable information. Thus, you should anonymize data by redacting workerIDs (and other identifiable information).

A note on best research practices. Do all analyses on anonymized data. (This is to prevent cases where others are unable to reproduce your analayses because it might rely somehow on identifiable information. If you start with anonymized data, your analyses would never use any of this information.)

• Beginner:

  • A gentle guide to MTurk

    • A good place to start

  • Guide to Mturk basics

    • Filled with lots of notes and tips (Thank you James/Jamil/Noah labs)

  • MTurkers are people (& the problem with common paradigms on MTurk)

  • Setting up simple experiments

    • Survey within MTurk platform

    • Qualtrics Link

      • My preferred method of directing turkers to Qualtrics

• Intermediate:

  • Turker Nation

    • Discussion board for Turkers

  • Creating a launch page to external site

    • advantage is that your study is embedded within MTurk, then opens a full page window for your task when required. Note launcher.html and task.html are separate.

  • Getting around the ridiculous extra 20% fee for 9+ participants

    • MTurk fee structure

HTML/CSS

• Beginner:

  • Interactive HTML/CSS Tutorial (Code Academy)

    • More structured course

  • Quick CSS-Specific Tutorial (Learn Layout)

  • CSS Reference (w3 schools)

    • Good resource with "Try it yourself" to see immediate output of code

• Intermediate:

  • CSS Positioning Tutorial (Code Academy)

    • Perhaps the most frustrating part of CSS

• Advanced

  • Bootstrap (Not of the statistical sort)

    • Generally better for creating full web-pages than one-off experiments, but can make introduction pages, survey websites etc. look good with minimal effort.

JavaScript

• Beginner:

  • Interactive JavaScript Tutorial (Code Academy)

  • JavaScript Reference (w3 schools)

• Intermediate:

  • jQuery (tool for making web pages responsive)

    • Interactive JQuery Tutorial (Code Academy)

    • JQuery Reference

  • JSPsych

    • Great resource if your experiment fits within your constraints- specifically created for web-based psych research, and works well with MTurk.

Github

• Beginner GitHub Tutorial (Try Git)

• Intermediate GitHub Tutorial (CodeAcademy)

• Advanced Cheatsheet (from official documentation)

• Git for Windows

R

• Beginner

  • Beginner R Tutorial (Good place for basic start)

  • http://r4ds.had.co.nz/ (Intro + Ch. 1- Will get you most of the basic competencies)

  • Swirl (+ Github repo) (Nice in R console learning)

• Intermediate/Advanced

  • (See package-specific tutorials below)

  • http://adv-r.had.co.nz/ (Great resource for those honing R skills)

  • Advanced guide to R’s many quirks (note: this is just for fun!)

• Everything else (because the needs are so diverse)

  • www.google.com

dplyr/tidyr

• Beginner:

  • http://r4ds.had.co.nz/wrangle-intro.html

  • dplyr Cheat sheet

  • dplyr Tutorial

  • dplyr/tidyr tutorial in Swirl ("Getting and Tidying Data")

  • Why tidy data?

    • Great way to understand purpose tidy data

• Everything else

  • https://www.google.com/#safe=off&q=how+do+I+use+tidyr

ggplot

• Beginner:

  • http://r4ds.had.co.nz/data-visualisation.html

  • Simple intro to ggplot basics

  • ggplot2 on Cookbook for R

  • ggplot2 docs

    • If you already know the kind of plot you want, they’ll usually have a load of nice examples to make it look right

• Everything else

  • https://www.google.com/#safe=off&q=how+to+ggplot2

lme4

• Intermediate

  • Mixed effects models

    • Good resource on why mixed effects models are beneficial and how to perform them in R

  • Keep it maximal

    • You don’t have to read the paper, but consider whether the maximal model is the model that makes the most sense. How to make the maximal model.

  • I understand mixed models - how do I set it up in lme4?

• Advanced

  • Nesting and a whole bunch of information on how to set up model

Functional Experiment Templates

Here are just a few templates tfhat may be helpful to build off of as you create your own online replications, with comments.

Simpler

Survey (Similar to Qualtrics with basic functionality) (+ PREVIEW)

• When you need to present stuff page by page and have people answer questions

Keypress Experiment based on Stimuli

• When participants see stuff and respond (quickly or slowly)

More Complex

Typing a list of things and Working Memory Task

• Complex randomization of presenting stimuli and responding in a list to be automatically checked

A whole lot of common research tasks

• Made by Stanford Psych Grad students - may require a few more tools (e.g. python), so users beware