Sundry notes and code bits for Stanford's Programming in Journalism class (COMM 177P/277P).
- Syllabus
- Assignments and Grading
- Bookshelf - recommended books, tutorials, cheatsheets, etc.
- Code solutions - A private repo containing solutions to class exercises.
- DataKit - overview and details on install and usage
- Getting Help - Resources and strategies for finding help.
- Glossary - technical terms used in class
- Python - overview, tutorials, etc.
- Technical setup and FAQ - recommended and required software (all free)
- Workflow advice - working on the command line, etc.
- Course overview - syllabus, grading, etc.
- Discuss the history of code and data analysis in journalism
- Preview of automation with Bash
- How to Succeed in this Course
- Final project overview and past examples
- The Owl, Problem Solving, and the Unix Workbench
- Bash intro
- Bash drill
- Tech setup through the Python section (stop before
Configurestep) - Assignment 0 - Unix practice and failed banks script
No class.
- Review Assignment 0 (failed banks) if everyone submitted...
- Bash drill - Yep. Again. This time using the "Blind-folded" and "Plain English" variations.
- Python Intro and resources links to misc docs/tutorials on this GitHub repo
- Python overview and coding contexts
- Python Syntax Crash Course
- Software installs -- everyone must have Python installed and GitHub configured by end of class.
- Assignment 1 - Python reading/practice and code challenge
- Assignment 2 - Python lists/dicts and DataKit overview
- Bash drill - Yep. Again. This time using the "Blind-folded" and "Plain English" variations.
- Questions about Assignment 1? It covered a lot of important fundamentals:
- Python interactive shell
- Expressions
- Basic data types
- Variables
- Flow control (conditions and blocks, if/else/elif etc.)
- Questions on Assignment 2 on Python lists/dicts?
- Dictionaries crash course
- Reading and writing text files - How to read text files
- Assignment 3 - Practical Python skills and Failed Banks in Python
- Bash drill - Yep. Again. This time using the "Blind-folded" and "Plain English" variations.
- Complete configuration steps for ssh and DataKit
- Reading and writing text files - Review how to write text files
- Libraries - Overview and practice practical skills
- Learn how to use code libraries included with Python or offered by third parties
- Learn how to download remote files
- Learn how to work with CSVs
- Election data code challenge
- Assignment 3 - Practical Python skills and Failed Banks in Python
- Questions about code libraries, how to download remote files and work with CSVs?
- Questions about code challenge?
- Automating workflows
- Overview of DataKit
- Complete Tech Setup configuration and install DataKit
- Continue work in-class on Assignment 3 - FDIC python script
- Wherefore, virtual environments and pipenv?
- The Art of Writing Functions
- Continue work on Assignment 3 - FDIC python script (now due by Sunday)
- Assignment 4 - Level up on Python functions, sorting (due by next class)
- Review/questions on functions/sorting (Assignment 4)
- Orchestrating functions with
main
- Orchestrating functions with
- APIs and the News overview and presentation
- Working with APIs
- Quakebot exercise - hands-on practice with a JSON feed
- Assignment 5 - Senate compromisers Python script
- Quakebot exercise solutions?
- Assignment 5 - Senate compromisers Python script
- Assignment 6 - Final project story idea pitches
- Web scraping for the news - a high-level overview of web scraping in a news context.
- Web scraping 101 - how to dissect a website, various obstacles to scraping and strategies for overcoming them
- Web scraping exercises - some websites to challenge your newfound scraping skills
- Web scraping resources - tutorials, etc.
- Data Pipelines with Modules
- API Services
- Refinitiv Entity Extraction
- Assignment 8 - First Python Notebook
- Assignment 9 - Data Journalism intro reading
- Project (extra credit): FDA Recall Entities - Scrape FDA recalls and extract entities using the OpenCalais/Refinitiv API
Class canceled due to President's Day.
- Discuss data journalism readings
- Whirlwind Tour of the Data Journalism Process
- Data Analysis with Jupyter and pandas Overview
- Coding workshop: Work on Final Project or FDA Recalls project
- Final project check-ins
- Coding workshop: Work on Final Project or FDA Recalls project
Class canceled due to NICAR 23 conference. Please work on your final projects if you're not attending.
- Final project check-ins
- Coding workshop: Work on Final Project or FDA Recalls project
- Coding workshop: Work on Final Project or FDA Recalls project
- Final Project presentations