Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor!: update duckdb, rusqlite and sqlite drivers to use <scheme>://<file> format #238

Merged
merged 1 commit into from
Dec 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ rsql --url "<url>" -- "<query>"
| cockroachdb (sqlx) | `cockroachdb://<user[:password>]@<host>[:<port>]/<database>` |
| csv (polars) | `csv://<file>[?has_header=<true/false>][&quote=<char>][&skip_rows=<n>]` |
| delimited (polars) | `delimited://<file>[?separator=<char>][&has_header=<true/false>][&quote=<char>][&skip_rows=<n>]` |
| duckdb | `duckdb://?<memory=true>[&file=<database_file>]` |
| duckdb | `duckdb://[<file>]` |
| json (polars) | `json://<file>` |
| jsonl (polars) | `jsonl://<file>` |
| libsql¹ | `libsql://<host>?[<memory=true>][&file=<database_file>][&auth_token=<token>]` |
Expand All @@ -84,9 +84,9 @@ rsql --url "<url>" -- "<query>"
| postgres | `postgres://<user>[:<password>]@<host>[:<port>]/<database>?<embedded=true>` |
| postgresql (sqlx) | `postgresql://<user>[:<password>]@<host>[:<port>]/<database>?<embedded=true>` |
| redshift (sqlx) | `redshift://<user[:password>]@<host>[:<port>]/<database>` |
| rusqlite | `rusqlite://?<memory=true>[&file=<database_file>]` |
| rusqlite | `rusqlite://[<file>]` |
| snowflake | `snowflake://<user>[:<token>]@<account>.snowflakecomputing.com/[?private_key_file=pkey_file&public_key_file=pubkey_file]` |
| sqlite (sqlx) | `sqlite://?<memory=true>[&file=<database_file>]` |
| sqlite (sqlx) | `sqlite://[<file>]` |
| sqlserver | `sqlserver://<user>[:<password>]@<host>[:<port>]/<database>` |
| tsv (polars) | `tsv://<file>[?has_header=<true/false>][&quote=<char>][&skip_rows=<n>]` |

Expand Down
Binary file added datasets/users.duckdb
Binary file not shown.
Binary file added datasets/users.sqlite3
Binary file not shown.
6 changes: 3 additions & 3 deletions rsql_cli/docs/src/chapter1/first-query.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ rsql --url "cockroachdb://<user[:password>]@<host>[:<port>]/<database>" -- "SELE
### DuckDB

```shell
rsql --url "duckdb://?memory=true" -- "SELECT version();"
rsql --url "duckdb://" -- "SELECT version();"
```

### LibSQL
Expand Down Expand Up @@ -53,7 +53,7 @@ rsql --url "redshift://<user[:password>]@<host>[:<port>]/<database>" -- "SELECT
### Rusqlite

```shell
rsql --url "rusqlite://?memory=true" -- "SELECT sqlite_version();"
rsql --url "rusqlite://" -- "SELECT sqlite_version();"
```

### Snowflake
Expand All @@ -71,5 +71,5 @@ rsql --url "snowflake://<user>[:<token>]@<account>.snowflakecomputing.com/" -- "
### Sqlite

```shell
rsql --url "sqlite://?memory=true" -- "SELECT sqlite_version();"
rsql --url "sqlite://" -- "SELECT sqlite_version();"
```
6 changes: 3 additions & 3 deletions rsql_cli/docs/src/chapter2/drivers/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ The drivers command displays the available database drivers.
| `cockroachdb` | CockroachDB driver provided by [SQLx](https://github.com/launchbadge/sqlx) | `cockroachdb://<user>[:<password>]@<host>[:<port>]/<database>` |
| `csv` | Comma Separated Value (CSV) file driver provided by [Polars](https://github.com/pola-rs/polars) | `csv://<file>[?has_header=<true/false>][&quote=<char>][&skip_rows=<n>]` |
| `delimited` | Delimited file driver provided by [Polars](https://github.com/pola-rs/polars) | `delimited://<file>[?separator=<char>][&has_header=<true/false>][&quote=<char>][&skip_rows=<n>]` |
| `duckdb` | DuckDB provided by [DuckDB](https://duckdb.org/) | `duckdb://?<memory=true>[&file=<database_file>]` |
| `duckdb` | DuckDB provided by [DuckDB](https://duckdb.org/) | `duckdb://[<file>]` |
| `json` | JSON file driver provided by [Polars](https://github.com/pola-rs/polars) | `json://<file>` |
| `jsonl` | JSONL file driver provided by [Polars](https://github.com/pola-rs/polars) | `jsonl://<file>` |
| `libsql` | LibSQL provided by [Turso](https://github.com/tursodatabase/libsql) | `libsql://<host>?[<memory=true>][&file=<database_file>][&auth_token=<token>]` |
Expand All @@ -27,9 +27,9 @@ The drivers command displays the available database drivers.
| `postgres` | PostgreSQL driver provided by [rust-postgres](https://github.com/sfackler/rust-postgres) | `postgres://<user>[:<password>]@<host>[:<port>]/<database>?<embedded=true>` |
| `postgresql` | PostgreSQL driver provided by [SQLx](https://github.com/launchbadge/sqlx) | `postgresql://<user>[:<password>]@<host>[:<port>]/<database>?<embedded=true>` |
| `redshift` | Redshift driver provided by [SQLx](https://github.com/launchbadge/sqlx) | `redshift://<user>[:<password>]@<host>[:<port>]/<database>` |
| `rusqlite` | SQLite provided by [Rusqlite](https://github.com/rusqlite/rusqlite?tab=readme-ov-file#rusqlite) | `rusqlite://?<memory=true>[&file=<database_file>]` |
| `rusqlite` | SQLite provided by [Rusqlite](https://github.com/rusqlite/rusqlite?tab=readme-ov-file#rusqlite) | `rusqlite://[<file>]` |
| `snowflake` | Snowflake provided by [Snowflake SQL API](https://docs.snowflake.com/en/developer-guide/sql-api/index) | `snowflake://<user>[:<token>]@<account>.snowflakecomputing.com/[?private_key_file=pkey_file&public_key_file=pubkey_file]` |
| `sqlite` | SQLite provided by [SQLx](https://github.com/launchbadge/sqlx) | `sqlite://?<memory=true>[&file=<database_file>]` |
| `sqlite` | SQLite provided by [SQLx](https://github.com/launchbadge/sqlx) | `sqlite://[<file>]` |
| `sqlserver` | SQL Server provided by [Tiberius](https://github.com/prisma/tiberius) | `sqlserver://<user>[:<password>]@<host>[:<port>]/<database>` |
| `tsv` | Tab Separated Value (TSV) file driver provided by [Polars](https://github.com/pola-rs/polars) | `tsv://<file>[?has_header=<true/false>][&quote=<char>][&skip_rows=<n>]` |

Expand Down
2 changes: 1 addition & 1 deletion rsql_core/benches/benchmarks/rusqlite.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ pub fn rusqlite_benchmark(criterion: &mut Criterion) {

async fn rusqlite() -> Result<i32> {
let args = ShellArgs {
url: "rusqlite://?memory=true".to_string(),
url: "rusqlite://".to_string(),
commands: vec!["SELECT 1".to_string()],
..ShellArgs::default()
};
Expand Down
2 changes: 1 addition & 1 deletion rsql_core/benches/benchmarks/sqlite.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ pub fn sqlite_benchmark(criterion: &mut Criterion) {

async fn sqlite() -> Result<i32> {
let args = ShellArgs {
url: "sqlite://?memory=true".to_string(),
url: "sqlite://".to_string(),
commands: vec!["SELECT 1".to_string()],
..ShellArgs::default()
};
Expand Down
2 changes: 1 addition & 1 deletion rsql_core/src/shell/args.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ use clap::Parser;
use clap_stdin::FileOrStdin;

#[cfg(feature = "driver-rusqlite")]
const DEFAULT_URL: &str = "rusqlite:://?memory=true";
const DEFAULT_URL: &str = "rusqlite://";

#[cfg(not(feature = "driver-rusqlite"))]
const DEFAULT_URL: &str = "";
Expand Down
43 changes: 15 additions & 28 deletions rsql_drivers/src/duckdb/driver.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
use crate::duckdb::metadata;
use crate::error::{Error, Result};
use crate::url::UrlExtension;
use crate::value::Value;
use crate::Error::UnsupportedColumnType;
use crate::{MemoryQueryResult, Metadata, QueryResult, StatementMetadata};
Expand All @@ -10,7 +11,6 @@ use duckdb::types::{TimeUnit, ValueRef};
use duckdb::Row;
use sqlparser::ast::Statement;
use sqlparser::dialect::{Dialect, DuckDbDialect};
use std::collections::HashMap;
use std::ops::Add;
use std::sync::{Arc, Mutex};
use std::time::Duration;
Expand Down Expand Up @@ -45,16 +45,10 @@ impl Connection {
#[expect(clippy::unused_async)]
pub(crate) async fn new(url: String) -> Result<Connection> {
let parsed_url = Url::parse(url.as_str())?;
let mut params: HashMap<String, String> = parsed_url.query_pairs().into_owned().collect();
let memory = params
.remove("memory")
.map_or(false, |value| value == "true");

let connection = if memory {
duckdb::Connection::open_in_memory()?
let connection = if let Ok(file_name) = parsed_url.to_file() {
duckdb::Connection::open(file_name)?
} else {
let file = params.get("file").map_or("", |value| value.as_str());
duckdb::Connection::open(file)?
duckdb::Connection::open_in_memory()?
};

Ok(Connection {
Expand Down Expand Up @@ -200,11 +194,12 @@ impl Connection {

#[cfg(test)]
mod test {
use crate::test::dataset_url;
use crate::{DriverManager, StatementMetadata, Value};
use chrono::{NaiveDate, NaiveDateTime, NaiveTime};
use indoc::indoc;

const DATABASE_URL: &str = "duckdb://?memory=true";
const DATABASE_URL: &str = "duckdb://";

#[tokio::test]
async fn test_driver_connect() -> anyhow::Result<()> {
Expand All @@ -217,34 +212,26 @@ mod test {

#[tokio::test]
async fn test_connection_interface() -> anyhow::Result<()> {
let database_url = dataset_url("duckdb", "users.duckdb");
let driver_manager = DriverManager::default();
let mut connection = driver_manager.connect(DATABASE_URL).await?;

let _ = connection
.execute("CREATE TABLE person (id INTEGER, name TEXT)")
.await?;
let mut connection = driver_manager.connect(&database_url).await?;

let rows = connection
.execute("INSERT INTO person (id, name) VALUES (1, 'foo')")
let mut query_result = connection
.query("SELECT id, name FROM users ORDER BY id")
.await?;
assert_eq!(rows, 1);

let mut query_result = connection.query("SELECT id, name FROM person").await?;
assert_eq!(query_result.columns().await, vec!["id", "name"]);
assert_eq!(
query_result.next().await,
Some(vec![Value::I32(1), Value::String("foo".to_string())])
Some(vec![Value::I32(1), Value::String("John Doe".to_string())])
);
assert_eq!(
query_result.next().await,
Some(vec![Value::I32(2), Value::String("Jane Smith".to_string())])
);
assert!(query_result.next().await.is_none());

connection.close().await?;

let db_metadata = connection.metadata().await?;
let schema = db_metadata
.current_schema()
.expect("expected at least one schema");
assert!(schema.tables().iter().any(|table| table.name() == "person"));

Ok(())
}

Expand Down
2 changes: 1 addition & 1 deletion rsql_drivers/src/duckdb/metadata.rs
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ async fn retrieve_indexes(connection: &mut dyn Connection, schema: &mut Schema)
mod test {
use crate::DriverManager;

const DATABASE_URL: &str = "duckdb://?memory=true";
const DATABASE_URL: &str = "duckdb://";

#[tokio::test]
async fn test_schema() -> anyhow::Result<()> {
Expand Down
27 changes: 12 additions & 15 deletions rsql_drivers/src/libsql/driver.rs
Original file line number Diff line number Diff line change
Expand Up @@ -36,26 +36,23 @@ pub(crate) struct Connection {
impl Connection {
pub(crate) async fn new(url: String) -> Result<Connection> {
let parsed_url = Url::parse(url.as_str())?;
let mut params: HashMap<String, String> = parsed_url.query_pairs().into_owned().collect();
let memory = params
.remove("memory")
.map_or(false, |value| value == "true");
let file = params.get("file").map_or("", |value| value.as_str());
let host = parsed_url.host();
let file_name = parsed_url.path();

let database = if memory {
Builder::new_local(":memory:").build().await?
} else if !file.is_empty() {
let db = Builder::new_local_replica(file).build().await?;
let frames = Frames::Vec(vec![]);
db.sync_frames(frames).await?;
db
} else {
let host = parsed_url.host().expect("Host is required").to_string();
let database = if let Some(host) = host {
let params: HashMap<String, String> = parsed_url.query_pairs().into_owned().collect();
let auth_token = params.get("auth_token").map_or("", |value| value.as_str());
let database_url = format!("libsql://{host}");
Builder::new_remote(database_url, auth_token.to_string())
.build()
.await?
} else if file_name.is_empty() {
Builder::new_local(":memory:").build().await?
} else {
let db = Builder::new_local_replica(file_name).build().await?;
let frames = Frames::Vec(vec![]);
db.sync_frames(frames).await?;
db
};

let connection = database.connect()?;
Expand Down Expand Up @@ -134,7 +131,7 @@ impl Debug for Connection {
mod test {
use crate::{DriverManager, Value};

const DATABASE_URL: &str = "libsql://?memory=true";
const DATABASE_URL: &str = "libsql://";

#[tokio::test]
async fn test_debug() -> anyhow::Result<()> {
Expand Down
2 changes: 1 addition & 1 deletion rsql_drivers/src/libsql/metadata.rs
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,7 @@ async fn retrieve_indexes(connection: &mut dyn Connection, schema: &mut Schema)
mod test {
use crate::DriverManager;

const DATABASE_URL: &str = "libsql://?memory=true";
const DATABASE_URL: &str = "libsql://";

#[tokio::test]
async fn test_schema() -> anyhow::Result<()> {
Expand Down
42 changes: 15 additions & 27 deletions rsql_drivers/src/rusqlite/driver.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
use crate::error::{Error, Result};
use crate::url::UrlExtension;
use crate::value::Value;
use crate::{sqlite, MemoryQueryResult, Metadata, QueryResult, StatementMetadata};
use anyhow::anyhow;
Expand All @@ -7,7 +8,6 @@ use rusqlite::types::ValueRef;
use rusqlite::Row;
use sqlparser::ast::Statement;
use sqlparser::dialect::{Dialect, SQLiteDialect};
use std::collections::HashMap;
use std::sync::{Arc, Mutex};
use url::Url;

Expand Down Expand Up @@ -40,16 +40,10 @@ impl Connection {
#[expect(clippy::unused_async)]
pub(crate) async fn new(url: String) -> Result<Connection> {
let parsed_url = Url::parse(url.as_str())?;
let mut params: HashMap<String, String> = parsed_url.query_pairs().into_owned().collect();
let memory = params
.remove("memory")
.map_or(false, |value| value == "true");

let connection = if memory {
rusqlite::Connection::open_in_memory()?
let connection = if let Ok(file_name) = parsed_url.to_file() {
rusqlite::Connection::open(file_name)?
} else {
let file = params.get("file").map_or("", |value| value.as_str());
rusqlite::Connection::open(file)?
rusqlite::Connection::open_in_memory()?
};

Ok(Connection {
Expand Down Expand Up @@ -149,9 +143,10 @@ impl Connection {

#[cfg(test)]
mod test {
use crate::test::dataset_url;
use crate::{DriverManager, Value};

const DATABASE_URL: &str = "rusqlite://?memory=true";
const DATABASE_URL: &str = "rusqlite://";

#[tokio::test]
async fn test_driver_connect() -> anyhow::Result<()> {
Expand All @@ -165,32 +160,25 @@ mod test {

#[tokio::test]
async fn test_connection_interface() -> anyhow::Result<()> {
let database_url = dataset_url("rusqlite", "users.sqlite3");
let driver_manager = DriverManager::default();
let mut connection = driver_manager.connect(DATABASE_URL).await?;
let mut connection = driver_manager.connect(&database_url).await?;

let _ = connection
.execute("CREATE TABLE person (id INTEGER, name TEXT)")
let mut query_result = connection
.query("SELECT id, name FROM users ORDER BY id")
.await?;

let rows = connection
.execute("INSERT INTO person (id, name) VALUES (1, 'foo')")
.await?;
assert_eq!(rows, 1);

let mut query_result = connection.query("SELECT id, name FROM person").await?;
assert_eq!(query_result.columns().await, vec!["id", "name"]);
assert_eq!(
query_result.next().await,
Some(vec![Value::I64(1), Value::String("foo".to_string())])
Some(vec![Value::I64(1), Value::String("John Doe".to_string())])
);
assert_eq!(
query_result.next().await,
Some(vec![Value::I64(2), Value::String("Jane Smith".to_string())])
);
assert!(query_result.next().await.is_none());

let db_metadata = connection.metadata().await?;
let schema = db_metadata
.current_schema()
.expect("expected at least one schema");
assert!(schema.tables().iter().any(|table| table.name() == "person"));

connection.close().await?;
Ok(())
}
Expand Down
Loading
Loading