Skip to content

Commit

Permalink
feat: add xml driver
Browse files Browse the repository at this point in the history
  • Loading branch information
@brianheineman
brianheineman committed Jan 6, 2025
1 parent 5a24836 commit 035f0ca
Show file tree
Hide file tree
Showing 13 changed files with 398 additions and 36 deletions.
1 change: 1 addition & 0 deletions Cargo.lock
View file

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ os_info = "3.9.0"
polars = "0.45.1"
polars-sql = "0.45.1"
postgresql_embedded = "0.17.3"
quick-xml = "0.37.1"
quick-xml = "0.37.2"
regex = "1.11.1"
reqwest = "0.12.8"
rusqlite = "0.30.0"
Expand Down
25 changes: 13 additions & 12 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -38,18 +38,18 @@ visit the [rsql](https://theseus-rs.github.io/rsql/rsql_cli/) site.

## Features

| Feature | |
|-----------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Databases | Arrow, Avro, CockroachDB, CSV, Delimited, DuckDB, JSON, JSONL, LibSQL (Turso), MariaDB, MySQL, Parquet, PostgreSQL, Redshift, Snowflake, SQLite3, SQL Server, TSV |
| Syntax Highlighting | ✅ |
| Result Highlighting | ✅ |
| Query Auto-completion | ✅ |
| History | ✅ |
| SQL File Execution | ✅ |
| Embedded PostgreSQL | ✅ |
| Output Formats | ascii, csv, expanded, html, json, jsonl, markdown, plain, psql, sqlite, tsv, unicode, xml, yaml |
| Localized Interface | 40+ languages¹ |
| Key Bindings | emacs, vi |
| Feature | |
|-----------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Databases | Arrow, Avro, CockroachDB, CSV, Delimited, DuckDB, JSON, JSONL, LibSQL (Turso), MariaDB, MySQL, Parquet, PostgreSQL, Redshift, Snowflake, SQLite3, SQL Server, TSV, XML |
| Syntax Highlighting | ✅ |
| Result Highlighting | ✅ |
| Query Auto-completion | ✅ |
| History | ✅ |
| SQL File Execution | ✅ |
| Embedded PostgreSQL | ✅ |
| Output Formats | ascii, csv, expanded, html, json, jsonl, markdown, plain, psql, sqlite, tsv, unicode, xml, yaml |
| Localized Interface | 40+ languages¹ |
| Key Bindings | emacs, vi |

¹ Computer translations; human translations welcome

Expand Down Expand Up @@ -90,6 +90,7 @@ rsql --url "<url>" -- "<query>"
| sqlite (sqlx) | `sqlite://[<file>]` |
| sqlserver | `sqlserver://<user>[:<password>]@<host>[:<port>]/<database>` |
| tsv (polars) | `tsv://<file>[?has_header=<true/false>][&quote=<char>][&skip_rows=<n>]` |
| xml | `xml://<file>` |

¹ the `file` driver will attempt to detect the type of file and automatically use the appropriate driver.
² `libsql` needs to be enabled with the `libsql` feature flag; it is disabled by default as it conflicts
Expand Down
11 changes: 11 additions & 0 deletions datasets/users.xml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
<?xml version="1.0" encoding="UTF-8" ?>
<data>
<user>
<id>1</id>
<name>John Doe</name>
</user>
<user>
<id>2</id>
<name>Jane Smith</name>
</user>
</data>
17 changes: 9 additions & 8 deletions rsql_cli/docs/src/chapter2/drivers/index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,27 +12,28 @@ The drivers command displays the available database drivers.

| Driver | Description | URL |
|---------------|--------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------|
| `arrow` | Arrow IPC file driver provided by [Polars](https://github.com/pola-rs/polars) | `arrow://<file>` |
| `avro` | Avro file driver provided by [Polars](https://github.com/pola-rs/polars) | `avro://<file>` |
| `arrow` | Arrow IPC provided by [Polars](https://github.com/pola-rs/polars) | `arrow://<file>` |
| `avro` | Avro provided by [Polars](https://github.com/pola-rs/polars) | `avro://<file>` |
| `cockroachdb` | CockroachDB driver provided by [SQLx](https://github.com/launchbadge/sqlx) | `cockroachdb://<user>[:<password>]@<host>[:<port>]/<database>` |
| `csv` | Comma Separated Value (CSV) file driver provided by [Polars](https://github.com/pola-rs/polars) | `csv://<file>[?has_header=<true/false>][&quote=<char>][&skip_rows=<n>]` |
| `delimited` | Delimited file driver provided by [Polars](https://github.com/pola-rs/polars) | `delimited://<file>[?separator=<char>][&has_header=<true/false>][&quote=<char>][&skip_rows=<n>]` |
| `csv` | Comma Separated Value (CSV) provided by [Polars](https://github.com/pola-rs/polars) | `csv://<file>[?has_header=<true/false>][&quote=<char>][&skip_rows=<n>]` |
| `delimited` | Delimited provided by [Polars](https://github.com/pola-rs/polars) | `delimited://<file>[?separator=<char>][&has_header=<true/false>][&quote=<char>][&skip_rows=<n>]` |
| `duckdb` | DuckDB provided by [DuckDB](https://duckdb.org/) | `duckdb://[<file>]` |
| `file` | File driver | `file://<file>` |
| `json` | JSON file driver provided by [Polars](https://github.com/pola-rs/polars) | `json://<file>` |
| `jsonl` | JSONL file driver provided by [Polars](https://github.com/pola-rs/polars) | `jsonl://<file>` |
| `json` | JSON provided by [Polars](https://github.com/pola-rs/polars) | `json://<file>` |
| `jsonl` | JSONL provided by [Polars](https://github.com/pola-rs/polars) | `jsonl://<file>` |
| `libsql` | LibSQL provided by [Turso](https://github.com/tursodatabase/libsql) | `libsql://<host>?[<memory=true>][&file=<database_file>][&auth_token=<token>]` |
| `mariadb` | MariaDB provided by [SQLx](https://github.com/launchbadge/sqlx) | `mariadb://<user>[:<password>]@<host>[:<port>]/<database>` |
| `mysql` | MySQL provided by [SQLx](https://github.com/launchbadge/sqlx) | `mysql://<user>[:<password>]@<host>[:<port>]/<database>` |
| `parquet` | Parquet file driver provided by [Polars](https://github.com/pola-rs/polars) | `parquet://<file>` |
| `parquet` | Parquet provided by [Polars](https://github.com/pola-rs/polars) | `parquet://<file>` |
| `postgres` | PostgreSQL driver provided by [rust-postgres](https://github.com/sfackler/rust-postgres) | `postgres://<user>[:<password>]@<host>[:<port>]/<database>?<embedded=true>` |
| `postgresql` | PostgreSQL driver provided by [SQLx](https://github.com/launchbadge/sqlx) | `postgresql://<user>[:<password>]@<host>[:<port>]/<database>?<embedded=true>` |
| `redshift` | Redshift driver provided by [SQLx](https://github.com/launchbadge/sqlx) | `redshift://<user>[:<password>]@<host>[:<port>]/<database>` |
| `rusqlite` | SQLite provided by [Rusqlite](https://github.com/rusqlite/rusqlite?tab=readme-ov-file#rusqlite) | `rusqlite://[<file>]` |
| `snowflake` | Snowflake provided by [Snowflake SQL API](https://docs.snowflake.com/en/developer-guide/sql-api/index) | `snowflake://<user>[:<token>]@<account>.snowflakecomputing.com/[?private_key_file=pkey_file&public_key_file=pubkey_file]` |
| `sqlite` | SQLite provided by [SQLx](https://github.com/launchbadge/sqlx) | `sqlite://[<file>]` |
| `sqlserver` | SQL Server provided by [Tiberius](https://github.com/prisma/tiberius) | `sqlserver://<user>[:<password>]@<host>[:<port>]/<database>` |
| `tsv` | Tab Separated Value (TSV) file driver provided by [Polars](https://github.com/pola-rs/polars) | `tsv://<file>[?has_header=<true/false>][&quote=<char>][&skip_rows=<n>]` |
| `tsv` | Tab Separated Value (TSV) provided by [Polars](https://github.com/pola-rs/polars) | `tsv://<file>[?has_header=<true/false>][&quote=<char>][&skip_rows=<n>]` |
| `xml` | Extensible Markup Language (XML) provided by [Polars](https://github.com/pola-rs/polars) | `xml://<file>` |

### Examples

Expand Down
2 changes: 2 additions & 0 deletions rsql_core/Cargo.toml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ all-drivers = [
"driver-sqlite",
"driver-sqlserver",
"driver-tsv",
"driver-xml",
]
driver-arrow = ["rsql_drivers/arrow"]
driver-avro = ["rsql_drivers/avro"]
Expand All @@ -100,6 +101,7 @@ driver-snowflake = ["rsql_drivers/snowflake"]
driver-sqlite = ["rsql_drivers/sqlite"]
driver-sqlserver = ["rsql_drivers/sqlserver"]
driver-tsv = ["rsql_drivers/tsv"]
driver-xml = ["rsql_drivers/xml"]
all-formats = [
"format-ascii",
"format-csv",
Expand Down
2 changes: 2 additions & 0 deletions rsql_core/src/commands/drivers.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,8 @@ mod tests {
"sqlserver",
#[cfg(feature = "driver-tsv")]
"tsv",
#[cfg(feature = "driver-xml")]
"xml",
];

let available_drivers = drivers.join(", ");
Expand Down
7 changes: 7 additions & 0 deletions rsql_drivers/Cargo.toml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ num-format = { workspace = true }
polars = { workspace = true, optional = true, features = ["avro", "ipc", "lazy", "json", "parquet", "polars-sql"] }
polars-sql = { workspace = true, optional = true }
postgresql_embedded = { workspace = true, optional = true }
quick-xml = { workspace = true, optional = true, features = ["serde"] }
regex = { workspace = true }
reqwest = { workspace = true, optional = true, features = ["json", "gzip"] }
rusqlite = { workspace = true, features = ["bundled-full"], optional = true }
Expand Down Expand Up @@ -87,6 +88,7 @@ all = [
"sqlite",
"sqlserver",
"tsv",
"xml",
]
default = []
arrow = [
Expand Down Expand Up @@ -171,6 +173,11 @@ tsv = [
"dep:polars",
"dep:polars-sql",
]
xml = [
"dep:polars",
"dep:polars-sql",
"dep:quick-xml",
]

[lints.clippy]
unwrap_used = "deny"
Expand Down
4 changes: 4 additions & 0 deletions rsql_drivers/src/driver.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,8 @@ impl Default for DriverManager {
drivers.add(Box::new(crate::sqlserver::Driver));
#[cfg(feature = "tsv")]
drivers.add(Box::new(crate::tsv::Driver));
#[cfg(feature = "xml")]
drivers.add(Box::new(crate::xml::Driver));

drivers
}
Expand Down Expand Up @@ -216,6 +218,8 @@ mod tests {
let driver_count = driver_count + 1;
#[cfg(feature = "tsv")]
let driver_count = driver_count + 1;
#[cfg(feature = "xml")]
let driver_count = driver_count + 1;

assert_eq!(driver_manager.drivers.len(), driver_count);
}
Expand Down
49 changes: 34 additions & 15 deletions rsql_drivers/src/file/driver.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -53,33 +53,52 @@ impl crate::Driver for Driver {
mod test {
use crate::test::dataset_url;
use crate::{DriverManager, Value};
use indoc::indoc;

#[tokio::test]
async fn test_file_drivers() -> anyhow::Result<()> {
let database_urls = vec![
dataset_url("file", "users.arrow"),
dataset_url("file", "users.avro"),
dataset_url("file", "users.csv"),
dataset_url("file", "users.duckdb"),
dataset_url("file", "users.json"),
dataset_url("file", "users.jsonl"),
dataset_url("file", "users.parquet"),
dataset_url("file", "users.sqlite3"),
dataset_url("file", "users.tsv"),
#[cfg(feature = "arrow")]
(dataset_url("file", "users.arrow"), None),
#[cfg(feature = "avro")]
(dataset_url("file", "users.avro"), None),
#[cfg(feature = "csv")]
(dataset_url("file", "users.csv"), None),
#[cfg(feature = "duckdb")]
(dataset_url("file", "users.duckdb"), None),
#[cfg(feature = "json")]
(dataset_url("file", "users.json"), None),
#[cfg(feature = "jsonl")]
(dataset_url("file", "users.jsonl"), None),
#[cfg(feature = "parquet")]
(dataset_url("file", "users.parquet"), None),
#[cfg(feature = "sqlite")]
(dataset_url("file", "users.sqlite3"), None),
#[cfg(feature = "tsv")]
(dataset_url("file", "users.tsv"), None),
#[cfg(feature = "xml")]
(
dataset_url("file", "users.xml"),
Some(indoc! {r"
WITH cte_user AS (
SELECT unnest(data.user) FROM users
)
SELECT user.* FROM cte_user
"}),
),
];
for database_url in database_urls {
test_file_driver(database_url.as_str()).await?;
for (database_url, sql) in database_urls {
test_file_driver(database_url.as_str(), sql).await?;
}
Ok(())
}

async fn test_file_driver(database_url: &str) -> anyhow::Result<()> {
async fn test_file_driver(database_url: &str, sql: Option<&str>) -> anyhow::Result<()> {
let sql = sql.unwrap_or("SELECT id, name FROM users ORDER BY id");
let driver_manager = DriverManager::default();
let mut connection = driver_manager.connect(database_url).await?;

let mut query_result = connection
.query("SELECT id, name FROM users ORDER BY id")
.await?;
let mut query_result = connection.query(sql).await?;

assert_eq!(query_result.columns().await, vec!["id", "name"]);
assert_eq!(
Expand Down
2 changes: 2 additions & 0 deletions rsql_drivers/src/lib.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,8 @@ mod test;
mod tsv;
mod url;
mod value;
#[cfg(feature = "xml")]
mod xml;

pub use connection::{
Connection, LimitQueryResult, MemoryQueryResult, MockConnection, QueryResult, StatementMetadata,
Expand Down
Loading

0 comments on commit 035f0ca

Please sign in to comment.