What the Hack Hackathon

Got Data? An Introduction to Using Spark on Azure Databricks

Use the challenges in this repo to get started using Spark in Azure Databricks.

0a. Start by following the Challenge0 to prepare your Azure environment.

0b. Download the all the Student files in the Files area in the Teams site for your hack.

After you successfully complete both steps listed as 0a and 0b, please complete the challenges in the following in order:

1. Challenge 1 - Getting Started with Spark In this challenge, you'll learn how to provision a Spark cluster in an Azure Databricks workspace, followed by interacting with the data using Python or Scala.

2. Challenge 2 - Introduction to Machine Learning. In this challenge, you'll be introduced to using Spark to train & evaluate a classification model.

3. Challenge 3 - Using Structured Streaming. In this challenge, you'll learn how to use Spark to process stream(s) of real-time data using IoT sensor data.