The objective of this lab is to setup your on-premise data warehouse environment for the hack. This will be your reference point for the migration.
- Complete setup of your environment
- Determine which SQL database offering is the best fit for the hack
- Estimate the size of your environment and the overall # of compute and storage nodes (No precision just concepts)
- Decision Tree for Analytics
- DWU Units
- Capacity Settings
- Capacity Limits SQL Dedicated Pools
- Synapse Analytics Best Practices & Field Guidance
- Azure Synapse Analytics Migration Guides
- Reference Architecture for Lambda Big Data Platforms
- Laptop computer as Development environment
- Azure Subscription
WWI runs their existing database platforms on-premise with SQL Server 2017. There are two databases samples for WWI. The first one is for their Line of Business application (OLTP) and the second is for their data warehouse (OLAP). You will need to setup both environments as our starting point in the migration.
-
Open your browser and login to your Azure Tenant. We plan to setup the Azure Services required for the What the Hack (WTH). In your portal, open the Azure Cloud Shell
-
Go into the cloud shell and select the subscription you plan to use for this WTH.
az account set --subscription {"Subscription Name"}
az account show
- Create a resource group to store the Modern Data Warehouse What the Hack. This will be the services for your source systems/environments. In Cloudshell, run this command
az group create --location eastus2 --name {"Resource Group Name"}
- In the Cloudshell, run this command to create a SQL Server instance and restore the databases. This will create an Azure Container Instance and restore the WideWorldImporters and WideWorldImoprtersDW databases. These two databases are your LOB databases for this hack.
az container create -g {Resource Group Name} --name mdwhackdb --image alexk002/sqlserver2019_demo:1 --cpu 2 --memory 7 --ports 1433 --ip-address Public
-
At the start of Challenge 1, reach out to your coach and they will share username and password for the LOB databases for this hack.
-
Upload your ARM templates into Azure CloudShell.
/Student/Challenges/Challenge0/ARM.
The files are parametersFile.json and template.json. Edit the parmeters file and replace any {} with information requested. -
Run the last command to setup Azure Data Factory, Azure SQL Server Instance and SSIS Runtime. This will build out for Challenge 1 the SSIS environment in Azure Data Factory.
az deployment group create --name final --resource-group {ENTER RESOURCE GROUP NAME} --template-file template.json --parameters parametersFile.json
-
Last step is to start your Azure Data Factory SSIS Runtime Service. Go to Connection pane in your Azure Data Factory service. The startup time is approximately 5 minutes.
-
Review the database catalog on the data warehouse for familiarity of the schema Reference document
-
Review ETL workflow to understand the data flow and architecture Reference document
