Skip to content

Commit

Permalink
[skip ci] update benchmark.md
Browse files Browse the repository at this point in the history
  • Loading branch information
wangxiaoying authored Nov 14, 2021
1 parent 7ae4d79 commit 21c6cd0
Showing 1 changed file with 13 additions and 5 deletions.
18 changes: 13 additions & 5 deletions Benchmark.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,14 @@ import connectorx as cx
cx.read_sql("connection string", "SELECT * FROM lineitem", partition_on="l_orderkey", partition_num=4)
```

Here are the baselines we compare againt:
* Pandas
* Modin
* Dask
* Turbodbc

Since Modin and Dask support parallel execution, we use the same number of cores (4) to run them. For Turbodbc, we use the result NumPy arrays to construct the final Pandas.DataFrame for a fair comparison.

## Postgres (db.m6g.4xlarge RDS)

## Time chart, lower is better.
Expand All @@ -88,7 +96,7 @@ cx.read_sql("connection string", "SELECT * FROM lineitem", partition_on="l_order

<p align="center"><img alt="memory chart" src="https://raw.githubusercontent.com/sfu-db/connector-agent/main/assets/pg-mem.png"/></p>

In conclusion, ConnectorX uses **3x** less memory and **13x** less time compared with pandas.
In conclusion, ConnectorX uses **3x** less memory and **13x** less time compared with Pandas.

## MySQL (db.m6g.4xlarge RDS)

Expand All @@ -101,7 +109,7 @@ In conclusion, ConnectorX uses **3x** less memory and **13x** less time compared

<p align="center"><img alt="memory chart" src="https://raw.githubusercontent.com/sfu-db/connector-agent/main/assets/mysql-mem.png"/></p>

In conclusion, ConnectorX uses **3x** less memory and **8x** less time compared with pandas.
In conclusion, ConnectorX uses **3x** less memory and **8x** less time compared with Pandas.


## SQLite (r5.4xlarge EC2 same instance)
Expand All @@ -116,7 +124,7 @@ In conclusion, ConnectorX uses **3x** less memory and **8x** less time compared

<p align="center"><img alt="memory chart" src="https://raw.githubusercontent.com/sfu-db/connector-agent/main/assets/sqlite-mem.png"/></p>

In conclusion, ConnectorX uses **2x** less memory and **5x** less time compared with pandas.
In conclusion, ConnectorX uses **2x** less memory and **5x** less time compared with Pandas.


## Oracle (db.r5.4xlarge RDS)
Expand All @@ -131,7 +139,7 @@ In conclusion, ConnectorX uses **2x** less memory and **5x** less time compared

<p align="center"><img alt="memory chart" src="https://raw.githubusercontent.com/sfu-db/connector-x/main/assets/oracle-mem.png"/></p>

In conclusion, ConnectorX uses **3x** less memory and **3x** less time compared with pandas.
In conclusion, ConnectorX uses **3x** less memory and **3x** less time compared with Pandas.



Expand All @@ -147,4 +155,4 @@ In conclusion, ConnectorX uses **3x** less memory and **3x** less time compared

<p align="center"><img alt="memory chart" src="https://raw.githubusercontent.com/sfu-db/connector-x/main/assets/mssql-mem.png"/></p>

In conclusion, ConnectorX uses **3x** less memory and **14x** less time compared with pandas.
In conclusion, ConnectorX uses **3x** less memory and **14x** less time compared with Pandas.

0 comments on commit 21c6cd0

Please sign in to comment.