spark-intro

Introduction to Big Data with Spark and Python

To run the demo:

1. Download and install these packages

• Virtual Box, http://www.virtualbox.org/
• Vagrant, http://www.vagrantup.com/

2. Fire up the virtual machine and log in

git clone https://github.com/dmkoch/spark-intro
cd spark-intro
vagrant up
vagrant ssh

3. Run pyspark from the shell

pyspark

4. View the IPython Notebook tutorial

• Browse to http://localhost:8001/notebooks/spark_tutorial_student.ipynb