Sundry notes and code bits for Stanford's Programming in Journalism class (winter quarter 2020).
- Syllabus
- Assignments
- Bookshelf - recommended books, tutorials, cheatsheets, etc.
- Code solutions - A private repo containing solutions to class exercises.
- Datakit - overview and details on install and usage
- Getting Help - Resources and strategies for finding help.
- Glossary - technical terms used in class
- Python - overview, tutorials, etc.
- Technical setup and FAQ - recommended and required software (all free)
- Workflow advice - working on the command line, etc.
- Overview of the course and syllabus.
- Discuss the history of code and data analysis in journalism.
- Dip our toes into Bash.
- Software install party (w00t!!)
- Assignment 0 - Tech setup; Unix practice and reading (Due: Mon. Jan 13th)
- The Owl, Problem Solving, and the Unix Workbench
- Group problem-solving exercise: Program your way to the Oval
- Hands-on bash practice with CLI murder mystery and bashcrawl
- Software installs, part deux (ssh keys and Python)
- Assignment 0 - Tech setup; Unix practice and reading (Due: Mon. Jan 13th)
- Bash drill
- Wrap up csvkit exercise from Power Tools for Data Wrangling
- Automating workflows and DataKit install
- Assignment 1 - Python ramp-up and Failed Banks shell script (Due: Wed. Jan. 22nd)
No class.
- Workflow Advice - In particular, working in the shell and with a code editor.
- Getting Help - How to ask G.O.O.D. questions and other advice/resources on getting help when programming.
- DataKit overview, including details on virtual environments, the benefits of git/GitHub, and daily workflow with DataKit.
- Submit Assignment 1 using DataKit.
- Assignment 2 - DataKit reading and Python lists and dicts (Due: Mon. Jan. 27th @ 1:30pm)
- Python intro and applying the basics
- Python overview and coding contexts
- The (i)Python interactive interpreter:
pip install ipython - Embracing errors
- Counting and filtering
- Reading and writing text files
- Assignment 3 - Python strings and Awesome Animals coding exercise (Due: Wed. Jan. 29th @ 1:30pm)
- Review Assignment 1 solutions, assuming everyone has submitted and time allows
- Working with libraries - standard library and 3rd-party libraries
- Reading and writing CSVs
- Downloading files
- Assignment 4 - Python functions and failed banks Python script (Due: Mon. Feb. 3rd @ 1:30pm)
- APIs and the News overview and presentation
- Working with APIs
- Portable (file) paths in Python
- Assignment 5 - Python sorting, portable paths and register for ProPublica API key (Due: Mon. Feb. 5th @ 1:30pm)
- The Art of Writing Functions
- Assignment 6 - Web basics (Due: Mon. Feb. 10th @ 1:30pm)
- Assignment 7 - Breaking ranks on impeachment Python script (Due: Wed. Feb. 12th @ 1:30pm)
- Web scraping for the news - a high-level overview of web scraping in a news context.
- Web scraping 101 - how to dissect a website, various obstacles to scraping and strategies for overcoming them
- Web scraping exercises - some websites to challenge your newfound scraping skills
- Web scraping resources - tutorials, etc.
- Assignment 8 - Web Scraping Ramp-up (Due: Wed. Feb. 12th @ 1:30pm)
- API Services
- OpenCalais Entity Extraction
- Data Pipelines with Modules
- Project: FDA Recall Entities - Scrape FDA recalls and extract entities using OpenCalais API (Due: Wed. Feb. 19th @ 1:30pm)
- Data Journalism - A Very Brief Tour
- Whirlwind Tour of the Data Journalism Process
- Data Analysis with Jupyter and pandas
- Assignment 9 - First Python Notebook
- Final Project - SF Data Analysis - Overview of final project and begin working on story ideas and data sleuthing/vetting.
- Story idea presentations are due Wed. Feb. 26th @ 1:30pm
- Final project presentations and related Jupyter notebooks are due Mon. March 9th @ 1:30pm.
- Groups will present story ideas for the final project.
- Work on project analysis, with opportunity for one-on-one help (time permitting).
Work on Final Project - SF Data Analysis.
No class due to NICAR 2020 conference.
Present and discuss Final Project - SF Data Analysis.
Wrap up final project presentations, followed by course review and reflection.