Stanford ~ Programming in Journalism

Sundry notes and code bits for Stanford's Programming in Journalism class (winter quarter 2020).

Important links

• Syllabus
• Assignments
• Bookshelf - recommended books, tutorials, cheatsheets, etc.
• Code solutions - A private repo containing solutions to class exercises.
• Datakit - overview and details on install and usage
• Getting Help - Resources and strategies for finding help.
• Glossary - technical terms used in class
• Python - overview, tutorials, etc.
• Technical setup and FAQ - recommended and required software (all free)
• Workflow advice - working on the command line, etc.

Class notes

Week 1

Day 1 - Course Intro

• Overview of the course and syllabus.
• Discuss the history of code and data analysis in journalism.
• Dip our toes into Bash.
• Software install party (w00t!!)
• Assignment 0 - Tech setup; Unix practice and reading (Due: Mon. Jan 13th)

Day 2 - The Owl, Problem Solving, and the Unix Workbench

• The Owl, Problem Solving, and the Unix Workbench
• Group problem-solving exercise: Program your way to the Oval
• Hands-on bash practice with CLI murder mystery and bashcrawl
• Software installs, part deux (ssh keys and Python)
• Assignment 0 - Tech setup; Unix practice and reading (Due: Mon. Jan 13th)

Week 2

Day 3 - Power Tools for Data Wrangling

• Power Tools for Data Wrangling

Day 4 - Power Tools cont.'d and Intro to DataKit

• Bash drill
• Wrap up csvkit exercise from Power Tools for Data Wrangling
• Automating workflows and DataKit install
• Assignment 1 - Python ramp-up and Failed Banks shell script (Due: Wed. Jan. 22nd)

Week 3

Day 5 - MLK Day

No class.

Day 6 - Failed Banks and Python

• Workflow Advice - In particular, working in the shell and with a code editor.
• Getting Help - How to ask G.O.O.D. questions and other advice/resources on getting help when programming.
• DataKit overview, including details on virtual environments, the benefits of git/GitHub, and daily workflow with DataKit.
• Submit Assignment 1 using DataKit.
• Assignment 2 - DataKit reading and Python lists and dicts (Due: Mon. Jan. 27th @ 1:30pm)

Week 4

Day 7 - Python intro and practical skills

• Python intro and applying the basics
  • Python overview and coding contexts
  • The (i)Python interactive interpreter: pip install ipython
  • Embracing errors
  • Counting and filtering
  • Reading and writing text files
  • Assignment 3 - Python strings and Awesome Animals coding exercise (Due: Wed. Jan. 29th @ 1:30pm)
  • Review Assignment 1 solutions, assuming everyone has submitted and time allows

Day 8 - Libraries, CSVs and remote files in Python

• Working with libraries - standard library and 3rd-party libraries
• Reading and writing CSVs
• Downloading files
• Assignment 4 - Python functions and failed banks Python script (Due: Mon. Feb. 3rd @ 1:30pm)

Week 5

Day 9 - APIs and the News

• APIs and the News overview and presentation
• Working with APIs
• Portable (file) paths in Python
• Assignment 5 - Python sorting, portable paths and register for ProPublica API key (Due: Mon. Feb. 5th @ 1:30pm)

Day 10 - Art of Functions and Senate Impeachment Script

• The Art of Writing Functions
• Assignment 6 - Web basics (Due: Mon. Feb. 10th @ 1:30pm)
• Assignment 7 - Breaking ranks on impeachment Python script (Due: Wed. Feb. 12th @ 1:30pm)

Week 6

Day 11 - Web scraping for the news

• Web scraping for the news - a high-level overview of web scraping in a news context.
• Web scraping 101 - how to dissect a website, various obstacles to scraping and strategies for overcoming them
• Web scraping exercises - some websites to challenge your newfound scraping skills
• Web scraping resources - tutorials, etc.
• Assignment 8 - Web Scraping Ramp-up (Due: Wed. Feb. 12th @ 1:30pm)

Day 12 - API Services and Data Pipelines with Modules

• API Services
• OpenCalais Entity Extraction
• Data Pipelines with Modules
• Project: FDA Recall Entities - Scrape FDA recalls and extract entities using OpenCalais API (Due: Wed. Feb. 19th @ 1:30pm)

Week 7

Day 13 - President's Day; no class

Day 14 - Data Journalism Overview and Intro to Jupyter

• Data Journalism - A Very Brief Tour
• Whirlwind Tour of the Data Journalism Process
• Data Analysis with Jupyter and pandas
• Assignment 9 - First Python Notebook

Week 8

Day 15 - Final Project

• Final Project - SF Data Analysis - Overview of final project and begin working on story ideas and data sleuthing/vetting.
• Story idea presentations are due Wed. Feb. 26th @ 1:30pm
• Final project presentations and related Jupyter notebooks are due Mon. March 9th @ 1:30pm.

Day 16 - Story idea presentations

• Groups will present story ideas for the final project.
• Work on project analysis, with opportunity for one-on-one help (time permitting).

Week 9

Day 17 - Data Workshop

Work on Final Project - SF Data Analysis.

Day 18 - class cancelled

No class due to NICAR 2020 conference.

Week 10

Day 19 - Final Project Presentations

Present and discuss Final Project - SF Data Analysis.

Day 20 - Presentations and course wrap-up

Wrap up final project presentations, followed by course review and reflection.