Skip to content

Commit

Permalink
36 using class for db types (#39)
Browse files Browse the repository at this point in the history 
* Big refactoring. Made databases into classes. Changed unit tests to reflect the same. Added more examples on README

* fix a mistake on readme
  • Loading branch information
@evangelosmeklis
evangelosmeklis authored Sep 25, 2024
1 parent 6fb39fb commit 51d1c71
Show file tree
Hide file tree
Showing 15 changed files with 503 additions and 461 deletions.
107 changes: 62 additions & 45 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,18 +12,6 @@
- **Formatted Output**: View data in a clean, formatted table or JSON format.
- **Pagination**: Efficiently handle large datasets by viewing data in manageable chunks.

## 🎬 peepDB in Action

Here's a quick demonstration of peepDB:

![peepDB Demo](images/demo.gif)

## 🖼️ peepDB stills

![peepDB example 2](images/peepdb_example2.png)

> **Note:** The above image reflect the commands used in the official release (v0.1.3)
## 📦 Installation

You can install peepDB directly from PyPI:
Expand All @@ -32,43 +20,39 @@ You can install peepDB directly from PyPI:
pip install peepdb
```

**Requirements:**
- Python 3.6 or higher
- pip (Python package installer)

> **Note:** If peepdb gives an error like "The term 'peepdb' is not recognized as the name of a cmdlet" remember to add the Python Scripts folder to your PATH in Windows.
## System Dependencies
## 🛠️ Usage

Before installing peepdb, ensure you have the following system dependencies:
peepDB uses a command-based structure for easier and more intuitive use. Here are the main commands with examples:

```bash
sudo apt-get update
sudo apt-get install libmariadb3 libmariadb-dev
```
### 1. Save Your Database Connection Details

Verify the installation by running:
```bash
peepdb --version
peepdb save <connection_name> --db-type [mysql/postgres/mariadb] --host <host> --user <user> --password <password> --database <database>
```

## 🛠️ Usage

peepDB uses a command-based structure for easier and more intuitive use. Here are the main commands:

### 1. Save Your Database Connection Details
**Important Note on Password Handling:**
The password is required as a command-line argument. While this is convenient for scripting, it's important to note that this method can be insecure as the password may be visible in your command history or to other users who can view your screen or process list.

Example:
```bash
peepdb save <connection_name> --db-type [mysql/postgres/mariadb] --host <host> --user <user> --database <database>
peepdb save myapp_db --db-type mysql --host localhost --user root --password my_secure_password --database myapp
```
You'll be prompted securely for the password.

For improved security, consider using environment variables or a configuration file to store sensitive information like passwords.

### 2. List Saved Connections

```bash
peepdb list
```

Example output:
```
Saved connections:
- myapp_db (mysql)
- analytics_db (postgres)
```

### 3. View Tables

View all tables:
Expand All @@ -81,20 +65,62 @@ View a specific table:
peepdb view <connection_name> --table <table_name>
```

Example:
```bash
peepdb view myapp_db --table users
```

Output:
```
Table: users
+----+----------+----------------------+
| id | username | email |
+====+==========+======================+
| 1 | johndoe | [email protected] |
| 2 | janedoe | [email protected] |
+----+----------+----------------------+
Page 1 of 1 (Total rows: 2)
```

### 4. Pagination

Use pagination to handle large datasets:
```bash
peepdb view <connection_name> --table <table_name> --page <page_number> --page-size <rows_per_page>
```

Example:
```bash
peepdb view myapp_db --table users --page 2 --page-size 50
```

### 5. Choose Output Format

Get output in JSON format:
```bash
peepdb view <connection_name> --format json
```

Example:
```bash
peepdb view myapp_db --table users --format json
```

Output:
```json
{
"users": {
"data": [
{"id": 1, "username": "johndoe", "email": "[email protected]"},
{"id": 2, "username": "janedoe", "email": "[email protected]"}
],
"page": 1,
"total_pages": 1,
"total_rows": 2
}
}
```

### 6. Remove Saved Connections

Remove a specific connection:
Expand All @@ -107,26 +133,17 @@ Remove all connections:
peepdb remove-all
```

For more detailed usage information on any command, use the `--help` option:
```bash
peepdb <command> --help
```

## 👨‍💻 For Developers

Please refer to our [Contributing Guide](CONTRIBUTING.md) for information on setting up the development environment, running tests, and contributing to peepDB.

## 🔒 Security

peepDB implements several security measures to protect your database connection details:

1. **Local Storage**: All connection details are stored locally on your machine, not on any remote servers.
2. **Encryption**: Connection details are encrypted before being stored, using the cryptography library.
3. **Secure Password Input**: Passwords are never shown in plain text and are input securely.
3. **Caution with Passwords**: While passwords are accepted as command-line arguments for convenience, users should be cautious about using this method in shared environments or situations where command history might be accessible to others.

However, please note that while we strive to implement best security practices, peepDB's security has not been verified by a third party. Users should exercise caution and follow general security best practices when using any tool that handles sensitive information.
## 🤝 Contributing

The encryption key is stored in your user directory (~/.peepdb/key.key). Keep this key safe and do not share it.
Contributions to peepDB are welcome! Please refer to our [Contributing Guide](CONTRIBUTING.md) for more information.

## 📜 License

Expand Down
32 changes: 22 additions & 10 deletions peepdb/cli.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,19 @@ def cli():
This tool allows you to quickly inspect database tables without writing SQL queries.
It supports MySQL, PostgreSQL, and MariaDB.
Commands:
- view: View database tables
- save: Save a new database connection
- list: List saved connections
- remove: Remove a specific saved connection
- remove-all: Remove all saved connections
Usage examples:
1. Save a new database connection:
peepdb save postgres1 --db-type postgres --host localhost --user postgres --password YourPassword --database peepdb_test
2. View all tables in a saved connection:
peepdb view postgres1
3. View a specific table with pagination:
peepdb view postgres1 --table users --page 2 --page-size 50
4. Remove a saved connection:
peepdb remove postgres1
Use 'peepdb COMMAND --help' for more information on a specific command.
"""
Expand Down Expand Up @@ -79,7 +86,7 @@ def view(connection_name, table, format, page, page_size):
@click.option('--db-type', type=click.Choice(['mysql', 'postgres', 'mariadb']), required=True, help='Database type')
@click.option('--host', required=True, help='Database host')
@click.option('--user', required=True, help='Database user')
@click.option('--password', required=True, prompt=True, hide_input=True, help='Database password')
@click.option('--password', required=True, help='Database password')
@click.option('--database', required=True, help='Database name')
def save(connection_name, db_type, host, user, password, database):
"""
Expand All @@ -88,7 +95,7 @@ def save(connection_name, db_type, host, user, password, database):
CONNECTION_NAME is the name to give to this saved connection.
Example:
peepdb save mydb --db-type mysql --host localhost --user root --database myapp
peepdb save postgres1 --db-type postgres --host localhost --user postgres --password YourPassword --database peepdb_test
"""
save_connection(connection_name, db_type, host, user, password, database)
click.echo(f"Connection '{connection_name}' saved successfully.")
Expand All @@ -104,7 +111,13 @@ def list():
Example:
peepdb list
"""
list_connections()
connections = list_connections()
if not connections:
click.echo("No saved connections.")
else:
click.echo("Saved connections:")
for name in connections:
click.echo(f"- {name}")


@cli.command()
Expand Down Expand Up @@ -144,6 +157,5 @@ def remove_all():
def main():
cli()


if __name__ == '__main__':
main()
115 changes: 25 additions & 90 deletions peepdb/core.py
View file
Original file line number Diff line number Diff line change
@@ -1,105 +1,40 @@
from peepdb.dbtypes import peepdb_mysql, peepdb_mariadb, peepdb_postgresql
import mysql.connector
import psycopg2
import pymysql
from decimal import Decimal
from datetime import date, time, datetime
from tabulate import tabulate
import logging
import math
from typing import Dict, Any
from .db import MySQLDatabase, PostgreSQLDatabase, MariaDBDatabase

logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger(__name__)


def connect_to_database(db_type, host, user, password, database, port=None, **kwargs):
logger.debug(f"Attempting to connect to {db_type} database '{database}' on host '{host}' with user '{user}'")
try:
if db_type == 'mysql':
conn = peepdb_mysql.connect_to_db(host=host, user=user, password=password, database=database,
port=port or 3306, **kwargs)
elif db_type == 'mariadb':
conn = peepdb_mariadb.connect_to_db(host=host, user=user, password=password, database=database,
port=port or 3306, **kwargs)
elif db_type == 'postgres':
conn = peepdb_postgresql.connect_to_db(host=host, user=user, password=password, database=database,
port=port or 5432, **kwargs)
else:
raise ValueError("Unsupported database type")
logger.debug("Connection successful")
return conn
except (mysql.connector.Error, psycopg2.Error, pymysql.Error) as e:
logger.error(f"Failed to connect to {db_type} database: {str(e)}")
raise ConnectionError(f"Failed to connect to {db_type} database: {str(e)}")


def fetch_tables(cursor, db_type):
tables = []
def connect_to_database(db_type: str, host: str, user: str, password: str, database: str, **kwargs):
if db_type == 'mysql':
tables = peepdb_mysql.fetch_tables(cursor)
elif db_type == 'mariadb':
tables = peepdb_mariadb.fetch_tables(cursor)
return MySQLDatabase(host, user, password, database, **kwargs)
elif db_type == 'postgres':
tables = peepdb_postgresql.fetch_tables(cursor)
return [table[0] for table in tables]


def view_table(cursor, table_name, page=1, page_size=100):
# Get total number of rows
cursor.execute(f"SELECT COUNT(*) FROM {table_name}")
total_rows = cursor.fetchone()[0]

# Calculate total pages
total_pages = math.ceil(total_rows / page_size)

# Ensure page is within bounds
page = max(1, min(page, total_pages))

# Calculate offset
offset = (page - 1) * page_size

# Fetch data for the current page
cursor.execute(f"SELECT * FROM {table_name} LIMIT {page_size} OFFSET {offset}")
columns = [col[0] for col in cursor.description]
rows = []
for row in cursor.fetchall():
row_dict = {}
for i, value in enumerate(row):
if isinstance(value, Decimal):
value = float(value)
elif isinstance(value, (date, time, datetime)):
value = value.isoformat()
row_dict[columns[i]] = value
rows.append(row_dict)

return {
'data': rows,
'page': page,
'total_pages': total_pages,
'total_rows': total_rows
}


def peep_db(db_type, host, user, password, database, table=None, format='table', page=1, page_size=100):
conn = connect_to_database(db_type, host, user, password, database)
cursor = conn.cursor()

if table:
result = {table: view_table(cursor, table, page, page_size)}
return PostgreSQLDatabase(host, user, password, database, **kwargs)
elif db_type == 'mariadb':
return MariaDBDatabase(host, user, password, database, **kwargs)
else:
tables = fetch_tables(cursor, db_type)
result = {table: view_table(cursor, table, page, page_size) for table in tables}
raise ValueError("Unsupported database type")

cursor.close()
conn.close()

if format == 'table':
return format_as_table(result)
else:
return result
def peep_db(db_type: str, host: str, user: str, password: str, database: str, table: str = None, format: str = 'table', page: int = 1, page_size: int = 100) -> Dict[str, Any]:
db = connect_to_database(db_type, host, user, password, database)
db.connect()
try:
if table:
result = {table: db.fetch_data(table, page, page_size)}
else:
tables = db.fetch_tables()
result = {table: db.fetch_data(table, page, page_size) for table in tables}

if format == 'table':
return format_as_table(result)
else:
return result
finally:
db.disconnect()

def format_as_table(data):
def format_as_table(data: Dict[str, Any]) -> str:
formatted_result = []
for table_name, table_data in data.items():
formatted_result.append(f"Table: {table_name}")
Expand All @@ -112,4 +47,4 @@ def format_as_table(data):
formatted_result.append(
f"Page {table_data['page']} of {table_data['total_pages']} (Total rows: {table_data['total_rows']})")
formatted_result.append("") # Add an empty line between tables
return "\n".join(formatted_result).strip()
return "\n".join(formatted_result).strip()
3 changes: 3 additions & 0 deletions peepdb/db/__init__.py
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
from .mysql import MySQLDatabase
from .postgresql import PostgreSQLDatabase
from .mariadb import MariaDBDatabase
Loading

0 comments on commit 51d1c71

Please sign in to comment.