-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Add support for logging - Improve the way connection arguments are passed - Add some DEBUG logging to make debugging easier
- Loading branch information
1 parent
0eb23b0
commit c14d86a
Showing
183 changed files
with
40,487 additions
and
4,455 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,3 @@ | ||
*.pyc | ||
dbconnector.egg-info/ | ||
example_db_options.secret |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,33 +1,66 @@ | ||
# MySQL-DBConnector # | ||
# MySQL-DBConnector | ||
|
||
A wrapper for the MySQL Python connector that provides additional resilience and functionality. | ||
|
||
## What is this repository for? ## | ||
**Latest version: 1.3.1** | ||
|
||
## About this repository | ||
|
||
* A wrapper for the MySQL Python connector that provides additional resilience and functionality. | ||
* Implements connection pooling that's more reliable. | ||
* Version 1.2.1 | ||
* Works with Python 2.7+ or 3.5+ | ||
* Developed and tested with Python 3.9, should work with 3.5+ | ||
|
||
## How do I get set up? ## | ||
## How do I get set up? | ||
|
||
* Make sure you have Git installed - [Download Git](https://git-scm.com/downloads) | ||
* Run `pip install git+https://github.com/SheffieldSolar/MySQL-DBConnector/` | ||
|
||
## Getting started ## | ||
I recommend using a MySQL options file to pass the connection parameters rather than hard-coding the DB credentials: | ||
## Usage | ||
Start by creating an instance of the `DBConnector` class, ideally using a context manager as this will ensure all connections are closed if an error is encountered or the code is interrupted: | ||
|
||
```Python | ||
from dbconnector import DBConnector | ||
``` | ||
|
||
The `DBConnector` class requires at least one argument: `connector_args`. This should be a dictionary of connection arguments, the full list of which can be found [here](https://dev.mysql.com/doc/connector-python/en/connector-python-connectargs.html). Most of these are optional, but you'll need to specify at least: | ||
|
||
```Python | ||
connector_args = { | ||
"user": "<your-username>", | ||
"password": "<your-password>", | ||
"database": "<your-database>", | ||
} | ||
``` | ||
|
||
Some of the connector args have default values which override the defaults in the mysql-connector-python library, they are: | ||
|
||
- `time_zone`: "UTC" | ||
- `connection_timeout`: 60 | ||
- `buffered`: True | ||
- `get_warnings`: True | ||
- `raise_on_warnings`: False | ||
- `use_pure`: True | ||
|
||
You can override these yourself if you prefer. | ||
|
||
Loading your password into your Python code is usually insecure, so I recommend using a MySQL options file to pass the connection parameters rather than hard-coding the DB credentials: | ||
```Python | ||
from dbconnector import DBConnector | ||
|
||
mysql_options_file = "mysql_options.txt" | ||
|
||
# Use the context manager to ensure DB connections are always safely closed | ||
# Always specify the session_tz to avoid issues with MySQL's timezone-naive datetimes or unwanted TZ conversions of timestamp fields! | ||
with DBConnector(mysql_options_file, session_tz="UTC") as dbc: | ||
# HINT: Use the context manager to ensure DB connections are always safely closed | ||
|
||
with DBConnector(connector_args={"option_files": mysql_options_file}) as dbc: | ||
# Selecting from the database (returns a list of tuples): | ||
# See https://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursor-fetchall.html | ||
# Note that using `select * from` is not recommended since the returned columns or their order | ||
# might change. | ||
data = dbc.query("SELECT * FROM test;") | ||
|
||
# Select from the database and convert to a Pandas DataFrame: | ||
data = dbc.query("SELECT col2, col2 FROM test;", df=True) | ||
|
||
# Inserting one row into the database: | ||
dbc.iud_query("insert into test (col1, col2, col3) values ('val1', 'val2', 'val3');") | ||
|
||
|
@@ -39,7 +72,9 @@ with DBConnector(mysql_options_file, session_tz="UTC") as dbc: | |
args = (1, 2, 3) | ||
result = dbc.proc("myProcedure", args) | ||
``` | ||
|
||
The file _'mysql_options.txt'_ should look something like this: | ||
|
||
``` | ||
[client] | ||
host = <your-server-hostname> | ||
|
@@ -49,32 +84,50 @@ database = <your-database> | |
``` | ||
|
||
It is also possible to pass connection parameters directly to the DBConnector class. This is not recommended but can be useful for testing/debugging: | ||
|
||
```Python | ||
from dbconnector import DBConnector | ||
|
||
db_config={"host": "<your-server-hostname>", "user": "<your-username>", | ||
"password": "<your-password>", "database": "<your-database>"} | ||
|
||
with DBConnector(db_config=db_config, session_tz="UTC") as dbc: | ||
# HINT: Always specify the time_zone arg to avoid issues with MySQL's timezone-naive datetimes or | ||
# unwanted TZ conversions of timestamp fields! | ||
connector_args = { | ||
"host": "<your-server-hostname>", | ||
"user": "<your-username>", | ||
"password": "<your-password>", | ||
"database": "<your-database>", | ||
"time_zone": "UTC" | ||
} | ||
|
||
with DBConnector(connector_args=connector_args) as dbc: | ||
# Do some DB stuff | ||
``` | ||
|
||
## Documentation ## | ||
The `dbconnector` module uses the Python `logging` module to log useful debugging messages, warnings and errors. You can set your preferred logging level using something like: | ||
|
||
```Python | ||
import os | ||
import logging | ||
|
||
fmt = "%(asctime)s %(module)s %(levelname)s %(message)s" | ||
logging.basicConfig(format=fmt, level=os.environ.get("LOGLEVEL", "DEBUG")) | ||
``` | ||
|
||
## Documentation | ||
|
||
* [https://sheffieldsolar.github.io/MySQL-DBConnector/](https://sheffieldsolar.github.io/MySQL-DBConnector/) | ||
- [https://sheffieldsolar.github.io/MySQL-DBConnector/](https://sheffieldsolar.github.io/MySQL-DBConnector/) | ||
|
||
## How do I update? ## | ||
## How do I update? | ||
|
||
* Run `pip install --upgrade git+https://github.com/SheffieldSolar/MySQL-DBConnector/` | ||
- Run `pip install --upgrade git+https://github.com/SheffieldSolar/MySQL-DBConnector/` | ||
|
||
## Who do I talk to? ## | ||
## Who do I talk to? | ||
|
||
* Jamie Taylor - [[email protected]](mailto:[email protected] "Email Jamie") - [SheffieldSolar](https://github.com/SheffieldSolar) | ||
- [Jamie Taylor](https://github.com/JamieTaylor-TUOS) - [[email protected]](mailto:[email protected] "Email Jamie") - [SheffieldSolar](https://github.com/SheffieldSolar) | ||
|
||
## Authors ## | ||
## Authors | ||
|
||
* **Jamie Taylor** - *Initial work* - [SheffieldSolar](https://github.com/SheffieldSolar) | ||
- [Jamie Taylor](https://github.com/JamieTaylor-TUOS) - [SheffieldSolar](https://github.com/SheffieldSolar) | ||
|
||
## License ## | ||
## License | ||
|
||
This project is licensed under the MIT License - see the [LICENSE.md](LICENSE.md) file for details |
Oops, something went wrong.