forked from apache/superset
-
Notifications
You must be signed in to change notification settings - Fork 11
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs(drivers): refresh guide on adding a db driver in docker (apache#…
- Loading branch information
Showing
1 changed file
with
21 additions
and
51 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -7,87 +7,57 @@ version: 1 | |
|
|
||
| ## Adding New Database Drivers in Docker | ||
|
|
||
| Superset requires a Python database driver to be installed for each additional type of database you | ||
| want to connect to. When setting up Superset locally via `docker compose`, the drivers and packages | ||
| contained in | ||
| [requirements.txt](https://github.com/apache/superset/blob/master/requirements.txt) and | ||
| [requirements-dev.txt](https://github.com/apache/superset/blob/master/requirements-dev.txt) | ||
| will be installed automatically. | ||
| Superset requires a Python database driver to be installed for each additional type of database you want to connect to. | ||
|
|
||
| In this section, we'll walk through how to install the MySQL connector library. The connector | ||
| library installation process is the same for all additional libraries and we'll end this section | ||
| with the recommended connector library for each database. | ||
| In this example, we'll walk through how to install the MySQL connector library. The connector library installation process is the same for all additional libraries. | ||
|
|
||
| ### 1. Determine the driver you need | ||
|
|
||
| To figure out how to install the [database driver](/docs/databases/installing-database-drivers) of your choice. | ||
| Consult the [list of database drivers](/docs/databases/installing-database-drivers) and find the PyPI package needed to connect to your database. In this example, we're connecting to a MySQL database, so we'll need the `mysqlclient` connector library. | ||
|
|
||
| In the example, we'll walk through the process of installing a MySQL driver in Superset. | ||
| ### 2. Install the driver in the container | ||
|
|
||
| ### 2. Install MySQL Driver | ||
| We need to get the `mysqlclient` library installed into the Superset docker container (it doesn't matter if it's installed on the host machine). We could enter the running container with `docker exec -it <container_name> bash` and run `pip install mysqlclient` there, but that wouldn't persist permanently. | ||
|
|
||
| As we are currently running inside of a Docker container via `docker compose`, we cannot simply run | ||
| `pip install mysqlclient` on our local shell and expect the drivers to be installed within the | ||
| Docker containers for superset. | ||
| To address this, the Superset `docker compose` deployment uses the convention of a `requirements-local.txt` file. All packages listed in this file will be installed into the container from PyPI at runtime. This file will be ignored by Git for the purposes of local development. | ||
|
|
||
| In order to address this, the Superset `docker compose` setup comes with a mechanism for you to | ||
| install packages locally, which will be ignored by Git for the purposes of local development. Please | ||
| follow these steps: | ||
|
|
||
| Create `requirements-local.txt` | ||
| Create the file `requirements-local.txt` in a subdirectory called `docker` that exists in the directory with your `docker-compose.yml` or `docker-compose-non-dev.yml` file. | ||
|
|
||
| ``` | ||
| # From the repo root... | ||
| # Run from the repo root: | ||
| touch ./docker/requirements-local.txt | ||
| ``` | ||
|
|
||
| Add the driver selected in step above: | ||
| Add the driver identified in step above. You can use a text editor or do it from the command line like: | ||
|
|
||
| ``` | ||
| echo "mysqlclient" >> ./docker/requirements-local.txt | ||
| ``` | ||
|
|
||
| Rebuild your local image with the new driver baked in: | ||
| **If you are running a stock (non-customized) Superset image**, you are done. Launch Superset with `docker compose -f docker-compose-non-dev.yml up` and the driver should be present. | ||
|
|
||
| ``` | ||
| docker compose build --force-rm | ||
| ``` | ||
| You can check its presence by entering the running container with `docker exec -it <container_name> bash` and running `pip freeze`. The PyPI package should be present in the printed list. | ||
|
|
||
| After the rebuild of the Docker images is complete (which may take a few minutes) you can relaunch using the following command: | ||
| **If you're running a customized docker image**, rebuild your local image with the new driver baked in: | ||
|
|
||
| ``` | ||
| docker compose up | ||
| docker compose build --force-rm | ||
| ``` | ||
|
|
||
| The other option is to start Superset via Docker Compose is using the recipe in `docker-compose-non-dev.yml`, which will use pre-built frontend assets and skip the building of front-end assets: | ||
|
|
||
| ``` | ||
| docker compose -f docker-compose-non-dev.yml pull | ||
| docker compose -f docker-compose-non-dev.yml up | ||
| ``` | ||
| After the rebuild of the Docker images is complete, relaunch Superset by running `docker compose up`. | ||
|
|
||
| ### 3. Connect to MySQL | ||
|
|
||
| Now that you've got a MySQL driver installed locally, you should be able to test it out. | ||
|
|
||
| We can now create a Datasource in Superset that can be used to connect to a MySQL instance. Assuming | ||
| your MySQL instance is running locally and can be accessed via localhost, use the following | ||
| connection string in “SQL Alchemy URI”, by going to Sources > Databases > + icon (to add a new | ||
| datasource) in Superset. | ||
| Now that you've got a MySQL driver installed in your container, you should be able to connect to your database via the Superset web UI. | ||
|
|
||
| For Docker running in Linux: | ||
| As an admin user, go to Settings -> Data: Database Connections and click the +DATABASE button. From there, follow the steps on the [Using Database Connection UI page](https://superset.apache.org/docs/databases/db-connection-ui). | ||
|
|
||
| ``` | ||
| mysql://mysqluser:mysqluserpassword@localhost/example?charset=utf8 | ||
| ``` | ||
| Consult the page for your specific database type in the Superset documentation to determine the connection string and any other parameters you need to input. For instance, on the [MySQL page](https://superset.apache.org/docs/databases/mysql), we see that the connection string to a local MySQL database differs depending on whether the setup is running on Linux or Mac. | ||
|
|
||
| For Docker running in OSX: | ||
| Click the “Test Connection” button, which should result in a popup message saying, "Connection looks good!". | ||
|
|
||
| ``` | ||
| mysql://mysqluser:[email protected]/example?charset=utf8 | ||
| ``` | ||
| ### 4. Troubleshooting | ||
|
|
||
| Then click “Test Connection”, which should give you an “OK” message. If not, please look at your | ||
| terminal for error messages, and reach out for help. | ||
| If the test fails, review your docker logs for error messages. Superset uses SQLAlchemy to connect to databases; to troubleshoot the connection string for your database, you might start Python in the Superset application container or host environment and try to connect directly to the desired database and fetch data. This eliminates Superset for the purposes of isolating the problem. | ||
|
|
||
| You can repeat this process for every database you want superset to be able to connect to. | ||
| Repeat this process for each different type of database you want Superset to be able to connect to. | ||