databricks-table-crud-fastapi

• This demo application demonstrates an implementation for CRUD operations on a databricks table via a REST Api.
• It is implemented with FastAPI and SQLAlchemy
• databricks-dbapi adds support for a databricks dialect for SQLAlchemy
• FastAPI automatically comes with an Interactive API documentation and exploration web user interfaces (SwaggerUI) and validation (via Pydantic) through class definitions

Databricks preperation

• First you need to initialize the databricks database + table and retrieve the connection information of your databricks instance for the connection string
• There is demo data crudapidata.csv and a databricks notebook databricks-crud-api-setup.py to create and load the databricks table in the repo
• Start your cluster
• Create (and load) data in the customer table with the notebook
• Create a personal access token (PAT)
• Retrieve the databricks cluster http-path for the connection string

Run FastAPI application

Configuration

• Create a '.env' configuration file in the root directory (where the README.md is located)
• Paste the content of the '.env.example' file in the root directory into '.env'

. \databricks-table-crud-flask-api
├── \sql_app
├── .env            < new File
├── .env.example
├── main.py
└── ...

• Adjust the parameters (Access Token, HttpPath) and save the '.env' file

.env

DB_ACCESS_TOKEN=your-databricks-personal-access-token-here
DB_DATABASE=crudapidb
DB_HTTP_PATH=your-databricks-httpath-here

If you already have another/newer Python version <> v3.7 installed:

• Download installer from https://www.python.org/downloads/release/python-379/
• Install without the option 'Add to path'
• Execute the following commands:

C:\source\databricks-table-crud-fastapi> pip install virtualenv
C:\source\databricks-table-crud-fastapi> virtualenv venv -p C:\Users\EMEAID\AppData\Local\Programs\Python\Python37\python.exe
C:\source\databricks-table-crud-fastapi> .\venv\Scripts\activate.ps1
(venv) C:\source\databricks-table-crud-fastapi> pip install https://download.lfd.uci.edu/pythonlibs/q4trcu4l/sasl-0.2.1-cp37-cp37m-win_amd64.whl
(venv) C:\source\databricks-table-crud-fastapi> pip install -r requirements.txt
(venv) C:\source\databricks-table-crud-fastapi> uvicorn main:app --reload

If you only have Python Version 3.7 installed:

• Execute the following commands:

C:\source\databricks-table-crud-fastapi> python -m venv venv
C:\source\databricks-table-crud-fastapi> .\venv\Scripts\activate.ps1
(venv) C:\source\databricks-table-crud-fastapi> pip install https://download.lfd.uci.edu/pythonlibs/q4trcu4l/sasl-0.2.1-cp37-cp37m-win_amd64.whl
(venv) C:\source\databricks-table-crud-fastapi> pip install -r requirements.txt
(venv) C:\source\databricks-table-crud-fastapi> uvicorn main:app --reload

Navigate to http://127.0.0.1:8000/docs
Have fun creating, reading, updating and deleting records of the databricks delta table 'customers' :)

Miscellaneous

Debugging FastAPI:

• Do not run it with uvicorn main:app --reload
• Instead follow this instructions: https://fastapi.tiangolo.com/tutorial/debugging/

Deployment to Azure App Service

• Under Settings > Configuration > General settings set the Startup Command to startup.sh (which is found in the root directory)
• See Configure a custom startup file for Python apps on Azure App Service