Skip to content

Commit

Permalink
Version 0.2.2 released!
Browse files Browse the repository at this point in the history
  • Loading branch information
Lavedonio committed Sep 28, 2020
1 parent 7451f2f commit 11cf856
Show file tree
Hide file tree
Showing 10 changed files with 192 additions and 30 deletions.
24 changes: 20 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ This Python library is an open source way to standardize and simplify connection
- [Version logs](#version-logs)

# Current release
**Version 0.2.1 (beta)**
**Version 0.2.2 (beta)**

# Prerequisites
1. Have a [Python 3.6 version or superior](https://www.python.org/downloads/) installed;
Expand Down Expand Up @@ -43,13 +43,15 @@ Use the files [secret_template.yml](https://github.com/Lavedonio/instackup/blob/
Location: local
Google:
secret_filename: file.json
BigQuery:
project_id:
project_name: "000000000000"
AWS:
access_key: AWSAWSAWSAWSAWSAWSAWS
secret_key: CcasldUYkfsadcSDadskfDSDAsdUYalf
Expand All @@ -68,13 +70,23 @@ RedShift:
password: masterpassword
port: 5439
PostgreSQL:
default:
dbname: postgres
user: postgres
host: localhost
password:
port: 5432
MySQL:
default:
dbname: mydb
host: localhost
user: root
password:
port: 3306
```
Save this file with `.yml` extension in a folder where you know the path won't be modified, like the Desktop folder (Example: `C:\Users\USER\Desktop\Credentials\secret.yml`).

Expand Down Expand Up @@ -214,16 +226,20 @@ Check the documentation by clicking in each topic.
- [SQLiteTool](https://github.com/Lavedonio/instackup/blob/master/docs/sql_tools.md#sqlitetool)
- [\_\_init\_\_(self, filename=None)](https://github.com/Lavedonio/instackup/blob/master/docs/sql_tools.md#__init__self-filenamenone)
- [describe_table(self, table, fetch_through_pandas=True, fail_silently=False)](https://github.com/Lavedonio/instackup/blob/master/docs/sql_tools.md#describe_tableself-table-fetch_through_pandastrue-fail_silentlyfalse)
- [PostgreSQLTool](https://github.com/Lavedonio/instackup/blob/master/docs/sql_tools.md#postgresqltool)
- [MySQLTool](https://github.com/Lavedonio/instackup/blob/master/docs/sql_tools.md#mysqltool)
- [\_\_init\_\_(self, connection='default')](https://github.com/Lavedonio/instackup/blob/master/docs/sql_tools.md#__init__self-connectiondefault)
- [describe_table(self, table, schema="public", fetch_through_pandas=True, fail_silently=False)](https://github.com/Lavedonio/instackup/blob/master/docs/sql_tools.md#describe_tableself-table-schemapublic-fetch_through_pandastrue-fail_silentlyfalse)
- [describe_table(self, table, fetch_through_pandas=True, fail_silently=False)](https://github.com/Lavedonio/instackup/blob/master/docs/sql_tools.md#describe_tableself-table-fetch_through_pandastrue-fail_silentlyfalse-1)
- [PostgreSQLTool](https://github.com/Lavedonio/instackup/blob/master/docs/sql_tools.md#postgresqltool)
- [\_\_init\_\_(self, connection='default')](https://github.com/Lavedonio/instackup/blob/master/docs/sql_tools.md#__init__self-connectiondefault-1)
- [describe_table(self, table, schema="public", fetch_through_pandas=True, fail_silently=False)](https://github.com/Lavedonio/instackup/blob/master/docs/sql_tools.md#describe_tableself-table-schemapublic-fetch_through_pandastrue-fail_silentlyfalse-2)
- [get_all_db_info(self, get_json_info=True, fetch_through_pandas=True, fail_silently=False)](https://github.com/Lavedonio/instackup/blob/master/docs/sql_tools.md#get_all_db_infoself-get_json_infotrue-fetch_through_pandastrue-fail_silentlyfalse)

# Version logs
See what changed in every version.

- Beta releases
- [Version 0.2.1](https://github.com/Lavedonio/instackup/blob/master/version_logs/v0.2.1-beta-current_release.md#version-021-beta) (current release)
- [Version 0.2.2](https://github.com/Lavedonio/instackup/blob/master/version_logs/v0.2.2-beta-current_release.md#version-022-beta) (current release)
- [Version 0.2.1](https://github.com/Lavedonio/instackup/blob/master/version_logs/v0.2.1-beta.md#version-021-beta)
- [Version 0.2.0](https://github.com/Lavedonio/instackup/blob/master/version_logs/v0.2.0-beta.md#version-020-beta)
- [Version 0.1.2](https://github.com/Lavedonio/instackup/blob/master/version_logs/v0.1.2-beta.md#version-012-beta)
- [Version 0.1.1](https://github.com/Lavedonio/instackup/blob/master/version_logs/v0.1.1-beta.md#version-011-beta)
Expand Down
11 changes: 10 additions & 1 deletion credentials/secret_blank.json
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,17 @@
"dbname": "postgres",
"user": "postgres",
"host": "localhost",
"password": "",
"password": null,
"port": 5432
}
},
"MySQL": {
"default": {
"dbname": "mydb",
"host": "localhost",
"user": "root",
"password": null,
"port": 3306
}
}
}
11 changes: 11 additions & 0 deletions credentials/secret_blank.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@

Location: local


Google:
secret_filename:

Expand All @@ -41,10 +42,20 @@ RedShift:
password:
port:


PostgreSQL:
default:
dbname: postgres
user: postgres
host: localhost
password:
port: 5432


MySQL:
default:
dbname: mydb
host: localhost
user: root
password:
port: 3306
13 changes: 11 additions & 2 deletions credentials/secret_template.json
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,19 @@
"PostgreSQL": {
"default": {
"dbname": "postgres",
"user": "postgres",
"host": "localhost",
"password": "",
"user": "postgres",
"password": null,
"port": 5432
}
},
"MySQL": {
"default": {
"dbname": "mydb",
"host": "localhost",
"user": "root",
"password": null,
"port": 3306
}
}
}
12 changes: 12 additions & 0 deletions credentials/secret_template.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,15 @@

Location: local


Google:
secret_filename: file.json

BigQuery:
project_id:
project_name: "000000000000"


AWS:
access_key: AWSAWSAWSAWSAWSAWSAWS
secret_key: CcasldUYkfsadcSDadskfDSDAsdUYalf
Expand All @@ -41,10 +43,20 @@ RedShift:
password: masterpassword
port: 5439


PostgreSQL:
default:
dbname: postgres
user: postgres
host: localhost
password:
port: 5432


MySQL:
default:
dbname: mydb
host: localhost
user: root
password:
port: 3306
92 changes: 90 additions & 2 deletions docs/sql_tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,12 @@ This is the documentation for the sql_tools module and all its contents, with us
- [SQLiteTool](#sqlitetool)
- [\_\_init\_\_(self, filename=None)](#__init__self-filenamenone)
- [describe_table(self, table, fetch_through_pandas=True, fail_silently=False)](#describe_tableself-table-fetch_through_pandastrue-fail_silentlyfalse)
- [PostgreSQLTool](#postgresqltool)
- [MySQLTool](#mysqltool)
- [\_\_init\_\_(self, connection='default')](#__init__self-connectiondefault)
- [describe_table(self, table, schema="public", fetch_through_pandas=True, fail_silently=False)](#describe_tableself-table-schemapublic-fetch_through_pandastrue-fail_silentlyfalse)
- [describe_table(self, table, fetch_through_pandas=True, fail_silently=False)](#describe_tableself-table-fetch_through_pandastrue-fail_silentlyfalse-1)
- [PostgreSQLTool](#postgresqltool)
- [\_\_init\_\_(self, connection='default')](#__init__self-connectiondefault-1)
- [describe_table(self, table, schema="public", fetch_through_pandas=True, fail_silently=False)](#describe_tableself-table-schemapublic-fetch_through_pandastrue-fail_silentlyfalse-2)
- [get_all_db_info(self, get_json_info=True, fetch_through_pandas=True, fail_silently=False)](#get_all_db_infoself-get_json_infotrue-fetch_through_pandastrue-fail_silentlyfalse)

# Module Contents
Expand Down Expand Up @@ -357,6 +360,91 @@ with SQLiteTool() as sl:
# other code
```

## MySQLTool
This class handle most of the interaction needed with PostgreSQL databases, so the base code becomes more readable and straightforward. This class inherits from [SQLTool](#sqltool), so its attributes and methods can (and will) be accessed from this class. Read the documentation of the base class for more info.

This class implements the with statement, so there are 2 ways of using it.

**1st way:**

```
from instackup.sql_tools import MySQLTool
with MySQLTool() as my:
# use my object to interact with PostgreSQL database
```

**2nd way:**

```
from instackup.sql_tools import MySQLTool
my = MySQLTool()
my.connect()
try:
# use my object to interact with PostgreSQL database
except Exception as e:
my.rollback()
raise e
else:
my.commit()
finally:
my.close_connection()
```

Easy to see that it is recommended (and easier) to use the first syntax.

### \_\_init\_\_(self, connection='default')
Initialization takes _connection_ parameter, that selects which connection to use. It has no return value.

The \_\_init\_\_ method doesn't actually opens the connection, but sets all values required by the connect method.

Usage example:
```
from instackup.sql_tools import MySQLTool
my = MySQLTool(connection='default')
```

### describe_table(self, table, fetch_through_pandas=True, fail_silently=False)
Returns all metadata from a specific table.

Usage example:
```
from instackup.sql_tools import MySQLTool
my = MySQLTool()
my.connect()
try:
# Returns a list of tuples containing the rows of the response
table = my.describe_table("users", fetch_through_pandas=False, fail_silently=True)
# Do something with table variable
except Exception as e:
my.rollback()
raise e
else:
my.commit()
finally:
# remember to close the connection later
my.close_connection()
# or
with MySQLTool() as my:
# Already connected, use my object in this context
# Returns a Pandas dataframe with all schema info of that specific schema.table
# To do operations with dataframe, you'll need to import pandas library
df = my.describe_table("airflow_logs")
# other code
```

## PostgreSQLTool
This class handle most of the interaction needed with PostgreSQL databases, so the base code becomes more readable and straightforward. This class inherits from [SQLTool](#sqltool), so its attributes and methods can (and will) be accessed from this class. Read the documentation of the base class for more info.

Expand Down
Loading

0 comments on commit 11cf856

Please sign in to comment.