The above button uses ARM template to deploy all necessary resources to the resource group. The ARM template however cannot deploy the following;
- Azure AD Application
- Databricks Cluster
- Databricks Notebooks
- Mount Azure Blob Storage containers with DBFS
- Key Vault secret for Databricks authentication token
- SQL Database Table
- Files in Azure Blob container
In order to address the above, we've created a setup bash script - deploy.sh
Make sure these tools are installed and added to the PATH
environment variable.
You need to manually create a Databricks personal access token. During script execution you're asked to provide Databricks workspace endpoint
"https://<Azure region>.azuredatabricks.net
and Databricks token. ADF also accesses Databricks clusters and notebooks using the same token.
The script uses environment variables to parameterize the deployment. They can be overwritten like this, prior to executing the script;
export RESOURCE_GROUP=data-manager
export AD_APP_PASSWORD=MyStrongADPaSSw0rd
...
In order to avoid invalid client secret error in Azure AD, please make sure to follow this guidance.