Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[WIP] feat: FastScan compressor objective #1896

Draft
wants to merge 9 commits into
base: develop
Choose a base branch
from
Draft

[WIP] feat: FastScan compressor objective #1896

wants to merge 9 commits into from

Conversation

lwwmanning
Copy link
Member

a reincarnation of #1068, but with a different objective function

@lwwmanning lwwmanning added the benchmark Run benchmarks on this branch label Jan 10, 2025
@lwwmanning lwwmanning changed the title feat: FastScan compressor objective [WIP] feat: FastScan compressor objective Jan 10, 2025
@github-actions github-actions bot removed the benchmark Run benchmarks on this branch label Jan 10, 2025
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 10, 2025 •
edited
Loading

Benchmarks: datafusion

Table of Results
name PR 39274fa base 047228b ratio (PR/base) unit
arrow/planning 950728 965248 0.984957 ns
arrow/exec 2.13842e+06 2.07074e+06 1.03269 ns
vortex-pushdown-compressed/planning 577580 581517 0.993229 ns
vortex-pushdown-compressed/exec 3.0595e+06 2.77273e+06 1.10343 ns
vortex-pushdown-uncompressed/planning 582658 584214 0.997338 ns
vortex-pushdown-uncompressed/exec 1.59284e+06 1.55333e+06 1.02543 ns
vortex-nopushdown-compressed/planning 953502 964135 0.988972 ns
vortex-nopushdown-compressed/exec 4.78194e+06 3.81891e+06 1.25217 ns
vortex-nopushdown-uncompressed/planning 959572 967367 0.991942 ns
vortex-nopushdown-uncompressed/exec 5.33537e+06 5.37322e+06 0.992956 ns

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 10, 2025 •
edited
Loading

Benchmarks: TPC-H

Table of Results
name PR 34204bb base 17e509f ratio (PR/base) unit
tpch_q01/arrow 575919970 6.00946e+08 0.958356 ns
tpch_q01/parquet 796086288 7.9837e+08 0.997139 ns
tpch_q01/vortex-file-compressed 535993916 5.75493e+08 0.931364 ns
tpch_q02/arrow 143933910 1.57736e+08 0.9125 ns
tpch_q02/parquet 180196246 1.8994e+08 0.948701 ns
tpch_q02/vortex-file-compressed 152476907 1.65979e+08 0.918649 ns
tpch_q03/arrow 177533211 1.85347e+08 0.957843 ns
tpch_q03/parquet 390624856 4.06733e+08 0.960397 ns
tpch_q03/vortex-file-compressed 210438865 2.48189e+08 0.847899 ns
tpch_q04/arrow 181588115 1.82555e+08 0.994705 ns
tpch_q04/parquet 221904522 2.21908e+08 0.999985 ns
tpch_q04/vortex-file-compressed 154432198 1.81937e+08 0.84882 ns
tpch_q05/arrow 328468744 3.52133e+08 0.932798 ns
tpch_q05/parquet 524948465 5.55224e+08 0.945472 ns
tpch_q05/vortex-file-compressed 368915322 4.12891e+08 0.893494 ns
tpch_q06/arrow 27435518 2.93784e+07 0.933868 ns
tpch_q06/parquet 153923246 1.55071e+08 0.992601 ns
tpch_q06/vortex-file-compressed 65746889 6.96467e+07 0.944006 ns
tpch_q07/arrow 643736326 6.95492e+08 0.925584 ns
tpch_q07/parquet 783169437 8.59157e+08 0.911555 ns
tpch_q07/vortex-file-compressed 636350159 7.02974e+08 0.905225 ns
tpch_q08/arrow 278135347 2.76481e+08 1.00598 ns
tpch_q08/parquet 558627002 5.70184e+08 0.979732 ns
tpch_q08/vortex-file-compressed 353592594 3.92998e+08 0.899732 ns
tpch_q09/arrow 498898997 5.43943e+08 0.91719 ns
tpch_q09/parquet 783641694 8.66511e+08 0.904364 ns
tpch_q09/vortex-file-compressed 577357851 6.67845e+08 0.864508 ns
tpch_q10/arrow 266049390 2.93465e+08 0.906581 ns
tpch_q10/parquet 527657367 5.37273e+08 0.982103 ns
tpch_q10/vortex-file-compressed 265480563 3.07779e+08 0.862568 ns
tpch_q11/arrow 140076655 1.51142e+08 0.926788 ns
tpch_q11/parquet 149876443 1.53391e+08 0.977088 ns
tpch_q11/vortex-file-compressed 135484545 1.42279e+08 0.952249 ns
tpch_q12/arrow 186006778 1.86159e+08 0.999181 ns
tpch_q12/parquet 333343157 3.32909e+08 1.0013 ns
tpch_q12/vortex-file-compressed 226369575 2.64552e+08 0.855671 ns
tpch_q13/arrow 176171075 1.75774e+08 1.00226 ns
tpch_q13/parquet 316307450 3.31905e+08 0.953006 ns
tpch_q13/vortex-file-compressed 191512933 1.9389e+08 0.987742 ns
tpch_q14/arrow 38848723 4.1257e+07 0.941627 ns
tpch_q14/parquet 231402800 2.40223e+08 0.963282 ns
tpch_q14/vortex-file-compressed 78604583 8.4838e+07 0.926526 ns
tpch_q15/arrow 71424593 6.95895e+07 1.02637 ns
tpch_q15/parquet 334819331 3.32465e+08 1.00708 ns
tpch_q15/vortex-file-compressed 142048756 1.38014e+08 1.02924 ns
tpch_q16/arrow 100947395 1.01097e+08 0.998524 ns
tpch_q16/parquet 117353417 1.16606e+08 1.00641 ns
tpch_q16/vortex-file-compressed 101035636 1.08472e+08 0.931446 ns
tpch_q17/arrow 643315634 6.66862e+08 0.964691 ns
tpch_q17/parquet 696885077 6.98973e+08 0.997013 ns
tpch_q17/vortex-file-compressed 616089088 6.20769e+08 0.992461 ns
tpch_q18/arrow 1175986814 1.17116e+09 1.00412 ns
tpch_q18/parquet 1391113703 1.38377e+09 1.00531 ns
tpch_q18/vortex-file-compressed 1175531332 1.20254e+09 0.977543 ns
tpch_q19/arrow 150991398 1.5075e+08 1.0016 ns
tpch_q19/parquet 420708348 4.1809e+08 1.00626 ns
tpch_q19/vortex-file-compressed 141893028 1.55752e+08 0.911019 ns
tpch_q20/arrow 179454016 1.84657e+08 0.971822 ns
tpch_q20/parquet 317559796 3.22929e+08 0.983374 ns
tpch_q20/vortex-file-compressed 223353471 2.2189e+08 1.00659 ns
tpch_q21/arrow 1013042537 1.01568e+09 0.997408 ns
tpch_q21/parquet 1158627121 1.13691e+09 1.0191 ns
tpch_q21/vortex-file-compressed 923679512 9.89813e+08 0.933186 ns
tpch_q22/arrow 81781796 7.99364e+07 1.02309 ns
tpch_q22/parquet 111311687 1.10038e+08 1.01158 ns
tpch_q22/vortex-file-compressed 88180032 8.42093e+07 1.04715 ns

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 10, 2025 •
edited
Loading

Benchmarks: random_access

Table of Results
name PR 34204bb base 17e509f ratio (PR/base) unit
random-access/vortex-tokio-local-disk 2.56362e+06 2.78278e+06 0.921243 ns
random-access/vortex-local-fs 3.30429e+06 3.50847e+06 0.941805 ns
random-access/parquet-tokio-local-disk 2.27036e+08 2.36752e+08 0.958965 ns

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 10, 2025 •
edited
Loading

Benchmarks: Clickbench

Table of Results
name PR 34204bb base 17e509f ratio (PR/base) unit
clickbench_q00/parquet 1786312 1.86409e+06 0.958277 ns
clickbench_q01/parquet 61981430 6.11919e+07 1.0129 ns
clickbench_q02/parquet 120975103 1.1832e+08 1.02244 ns
clickbench_q03/parquet 85257505 8.35737e+07 1.02015 ns
clickbench_q04/parquet 691048790 6.59305e+08 1.04815 ns
clickbench_q05/parquet 837241674 8.26945e+08 1.01245 ns
clickbench_q06/parquet 1918102 2.00307e+06 0.957582 ns
clickbench_q07/parquet 64898082 6.15563e+07 1.05429 ns
clickbench_q08/parquet 776082264 7.44751e+08 1.04207 ns
clickbench_q09/parquet 1080855363 1.03577e+09 1.04353 ns
clickbench_q10/parquet 258990121 2.49424e+08 1.03835 ns
clickbench_q11/parquet 296641405 2.98203e+08 0.994763 ns
clickbench_q12/parquet 868999417 8.34453e+08 1.0414 ns
clickbench_q13/parquet 1084385220 1.09983e+09 0.985959 ns
clickbench_q14/parquet 838782335 8.33085e+08 1.00684 ns
clickbench_q15/parquet 781873235 7.45015e+08 1.04947 ns
clickbench_q16/parquet 1685634652 1.63526e+09 1.03081 ns
clickbench_q17/parquet 1461497275 1.42116e+09 1.02838 ns
clickbench_q18/parquet 3090838825 3.0016e+09 1.02973 ns
clickbench_q19/parquet 66298039 6.53163e+07 1.01503 ns
clickbench_q20/parquet 1220085306 1.22821e+09 0.993384 ns
clickbench_q21/parquet 1402147680 1.44596e+09 0.969697 ns
clickbench_q22/parquet 2468491825 2.45662e+09 1.00483 ns
clickbench_q23/parquet 8450986266 8.24401e+09 1.02511 ns
clickbench_q24/parquet 542377612 5.26871e+08 1.02943 ns
clickbench_q25/parquet 515331555 5.13589e+08 1.00339 ns
clickbench_q26/parquet 602569676 5.83772e+08 1.0322 ns
clickbench_q27/parquet 1676162851 1.61951e+09 1.03498 ns
clickbench_q28/parquet 11515094542 1.1504e+10 1.00097 ns
clickbench_q29/parquet 423021393 4.28963e+08 0.986149 ns
clickbench_q30/parquet 793045408 7.6284e+08 1.0396 ns
clickbench_q31/parquet 813909190 8.47803e+08 0.960021 ns
clickbench_q32/parquet 2839926837 2.74172e+09 1.03582 ns
clickbench_q33/parquet 3100338188 2.91379e+09 1.06402 ns
clickbench_q34/parquet 2903461560 2.81516e+09 1.03137 ns
clickbench_q35/parquet 853657689 8.4441e+08 1.01095 ns
clickbench_q36/parquet 174985815 1.68249e+08 1.04004 ns
clickbench_q37/parquet 87285320 8.4064e+07 1.03832 ns
clickbench_q38/parquet 113132413 1.11342e+08 1.01608 ns
clickbench_q39/parquet 328368542 3.22905e+08 1.01692 ns
clickbench_q40/parquet 51837379 4.87067e+07 1.06428 ns
clickbench_q41/parquet 48459183 4.77999e+07 1.01379 ns
clickbench_q42/parquet 68804668 6.78424e+07 1.01418 ns
clickbench_q00/vortex-file-compressed 2011446 2.15782e+06 0.932166 ns
clickbench_q01/vortex-file-compressed 27286292 2.7457e+07 0.993783 ns
clickbench_q02/vortex-file-compressed 86067182 8.87254e+07 0.97004 ns
clickbench_q03/vortex-file-compressed 97257047 8.2996e+07 1.17183 ns
clickbench_q04/vortex-file-compressed 650691165 6.47827e+08 1.00442 ns
clickbench_q05/vortex-file-compressed 652416756 6.67005e+08 0.978129 ns
clickbench_q06/vortex-file-compressed 2110498 2.14938e+06 0.981908 ns
clickbench_q07/vortex-file-compressed 44887176 4.54052e+07 0.98859 ns
clickbench_q08/vortex-file-compressed 776510979 7.74261e+08 1.00291 ns
clickbench_q09/vortex-file-compressed 968262845 9.65313e+08 1.00306 ns
clickbench_q10/vortex-file-compressed 158977966 2.04794e+08 0.776283 ns
clickbench_q11/vortex-file-compressed 188019277 2.34738e+08 0.800974 ns
clickbench_q12/vortex-file-compressed 546575514 5.93892e+08 0.920328 ns
clickbench_q13/vortex-file-compressed 805280405 9.09702e+08 0.885213 ns
clickbench_q14/vortex-file-compressed 552592299 5.94292e+08 0.929833 ns
clickbench_q15/vortex-file-compressed 798993852 7.80336e+08 1.02391 ns
clickbench_q16/vortex-file-compressed 1440810349 1.43987e+09 1.00065 ns
clickbench_q17/vortex-file-compressed 1393261998 1.33787e+09 1.04141 ns
clickbench_q18/vortex-file-compressed 3068520890 3.0019e+09 1.02219 ns
clickbench_q19/vortex-file-compressed 54445189 4.31595e+07 1.26149 ns
clickbench_q20/vortex-file-compressed 510335179 5.00262e+08 1.02014 ns
clickbench_q21/vortex-file-compressed 786265927 7.84526e+08 1.00222 ns
clickbench_q22/vortex-file-compressed 1907486250 1.90853e+09 0.999453 ns
clickbench_q23/vortex-file-compressed 3691236225 3.84579e+09 0.959812 ns
clickbench_q24/vortex-file-compressed 271604318 3.4935e+08 0.777456 ns
clickbench_q25/vortex-file-compressed 243626102 3.28715e+08 0.741148 ns
clickbench_q26/vortex-file-compressed 339156389 4.24232e+08 0.799459 ns
clickbench_q27/vortex-file-compressed 1382017896 1.37082e+09 1.00817 ns
clickbench_q28/vortex-file-compressed 10645660462 1.06535e+10 0.999268 ns
clickbench_q29/vortex-file-compressed 727209547 7.327e+08 0.992507 ns
clickbench_q30/vortex-file-compressed 477187916 5.84679e+08 0.816153 ns
clickbench_q31/vortex-file-compressed 502617006 6.24929e+08 0.804278 ns
clickbench_q32/vortex-file-compressed 2862184114 2.92201e+09 0.979526 ns
clickbench_q33/vortex-file-compressed 2256318365 2.30941e+09 0.97701 ns
clickbench_q34/vortex-file-compressed 2295767260 2.30368e+09 0.996564 ns
clickbench_q35/vortex-file-compressed 1008701489 9.87771e+08 1.02119 ns
clickbench_q36/vortex-file-compressed 134075202 6.17221e+07 2.17224 ns
clickbench_q37/vortex-file-compressed 66460568 5.35921e+07 1.24012 ns
clickbench_q38/vortex-file-compressed 77462121 4.72327e+07 1.64001 ns
clickbench_q39/vortex-file-compressed 251682033 1.05517e+08 2.38523 ns
clickbench_q40/vortex-file-compressed 33979838 3.17876e+07 1.06897 ns
clickbench_q41/vortex-file-compressed 32947375 3.24583e+07 1.01507 ns
clickbench_q42/vortex-file-compressed 55272756 4.58104e+07 1.20656 ns

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 10, 2025 •
edited
Loading

Benchmarks: compress

Table of Results
name PR 34204bb base 17e509f ratio (PR/base) unit
compress time/wide table cols=10 chunks=1 rows=1000 4.1801e+06 4.13681e+06 1.01046 ns
compress time/wide table cols=10 chunks=1 rows=1000 throughput 0.0287414 0.0290422 0.989644 bytes/ns
parquet_rs-zstd compress time/wide table cols=10 chunks=1 rows=1000 735536 735872 0.999544 ns
parquet_rs-zstd compress time/wide table cols=10 chunks=1 rows=1000 throughput 0.163339 0.163265 1.00046 bytes/ns
decompress time/wide table cols=10 chunks=1 rows=1000 417503 415815 1.00406 ns
decompress time/wide table cols=10 chunks=1 rows=1000 throughput 0.287763 0.288932 0.995956 bytes/ns
parquet_rs-zstd decompress time/wide table cols=10 chunks=1 rows=1000 268664 271495 0.98957 ns
parquet_rs-zstd decompress time/wide table cols=10 chunks=1 rows=1000 throughput 0.447184 0.44252 1.01054 bytes/ns
compress time/wide table cols=100 chunks=1 rows=1000 4.16122e+07 4.17583e+07 0.9965 ns
compress time/wide table cols=100 chunks=1 rows=1000 throughput 0.0288671 0.028766 1.00351 bytes/ns
parquet_rs-zstd compress time/wide table cols=100 chunks=1 rows=1000 7.84728e+06 7.72231e+06 1.01618 ns
parquet_rs-zstd compress time/wide table cols=100 chunks=1 rows=1000 throughput 0.153075 0.155552 0.984075 bytes/ns
decompress time/wide table cols=100 chunks=1 rows=1000 4.02541e+06 4.05204e+06 0.993428 ns
decompress time/wide table cols=100 chunks=1 rows=1000 throughput 0.29841 0.296449 1.00662 bytes/ns
parquet_rs-zstd decompress time/wide table cols=100 chunks=1 rows=1000 2.71253e+06 2.78293e+06 0.974704 ns
parquet_rs-zstd decompress time/wide table cols=100 chunks=1 rows=1000 throughput 0.442841 0.431639 1.02595 bytes/ns
compress time/wide table cols=1000 chunks=1 rows=1000 4.20566e+08 4.13853e+08 1.01622 ns
compress time/wide table cols=1000 chunks=1 rows=1000 throughput 0.0285616 0.0290248 0.984038 bytes/ns
parquet_rs-zstd compress time/wide table cols=1000 chunks=1 rows=1000 9.01838e+07 9.30139e+07 0.969573 ns
parquet_rs-zstd compress time/wide table cols=1000 chunks=1 rows=1000 throughput 0.133195 0.129142 1.03138 bytes/ns
decompress time/wide table cols=1000 chunks=1 rows=1000 7.07969e+07 7.21175e+07 0.981687 ns
decompress time/wide table cols=1000 chunks=1 rows=1000 throughput 0.169669 0.166562 1.01865 bytes/ns
parquet_rs-zstd decompress time/wide table cols=1000 chunks=1 rows=1000 3.05572e+07 3.17535e+07 0.962328 ns
parquet_rs-zstd decompress time/wide table cols=1000 chunks=1 rows=1000 throughput 0.393099 0.37829 1.03915 bytes/ns
compress time/wide table cols=10 chunks=50 rows=1000 8.32872e+06 8.41381e+06 0.989887 ns
compress time/wide table cols=10 chunks=50 rows=1000 throughput 0.015184 0.0150304 1.01022 bytes/ns
parquet_rs-zstd compress time/wide table cols=10 chunks=50 rows=1000 1.10268e+06 1.12448e+06 0.98061 ns
parquet_rs-zstd compress time/wide table cols=10 chunks=50 rows=1000 throughput 0.114687 0.112463 1.01977 bytes/ns
decompress time/wide table cols=10 chunks=50 rows=1000 436121 435465 1.00151 ns
decompress time/wide table cols=10 chunks=50 rows=1000 throughput 0.289972 0.290409 0.998496 bytes/ns
parquet_rs-zstd decompress time/wide table cols=10 chunks=50 rows=1000 278820 280492 0.99404 ns
parquet_rs-zstd decompress time/wide table cols=10 chunks=50 rows=1000 throughput 0.453565 0.450862 1.006 bytes/ns
compress time/wide table cols=100 chunks=50 rows=1000 8.63561e+07 8.97195e+07 0.962512 ns
compress time/wide table cols=100 chunks=50 rows=1000 throughput 0.0145961 0.0140489 1.03895 bytes/ns
parquet_rs-zstd compress time/wide table cols=100 chunks=50 rows=1000 1.22055e+07 1.24653e+07 0.979154 ns
parquet_rs-zstd compress time/wide table cols=100 chunks=50 rows=1000 throughput 0.10327 0.101118 1.02129 bytes/ns
decompress time/wide table cols=100 chunks=50 rows=1000 4.21672e+06 4.19257e+06 1.00576 ns
decompress time/wide table cols=100 chunks=50 rows=1000 throughput 0.29892 0.300642 0.994271 bytes/ns
parquet_rs-zstd decompress time/wide table cols=100 chunks=50 rows=1000 2.86042e+06 2.85514e+06 1.00185 ns
parquet_rs-zstd decompress time/wide table cols=100 chunks=50 rows=1000 throughput 0.440656 0.441472 0.998153 bytes/ns
compress time/wide table cols=1000 chunks=50 rows=1000 9.5167e+08 9.59003e+08 0.992354 ns
compress time/wide table cols=1000 chunks=50 rows=1000 throughput 0.0132404 0.0131391 1.0077 bytes/ns
parquet_rs-zstd compress time/wide table cols=1000 chunks=50 rows=1000 1.71144e+08 1.77513e+08 0.964122 ns
parquet_rs-zstd compress time/wide table cols=1000 chunks=50 rows=1000 throughput 0.0736249 0.0709834 1.03721 bytes/ns
decompress time/wide table cols=1000 chunks=50 rows=1000 7.95626e+07 7.99273e+07 0.995437 ns
decompress time/wide table cols=1000 chunks=50 rows=1000 throughput 0.158372 0.157649 1.00458 bytes/ns
parquet_rs-zstd decompress time/wide table cols=1000 chunks=50 rows=1000 3.50353e+07 3.64635e+07 0.960832 ns
parquet_rs-zstd decompress time/wide table cols=1000 chunks=50 rows=1000 throughput 0.35965 0.345564 1.04076 bytes/ns
compress time/taxi 1.41101e+09 1.59122e+09 0.886747 ns
compress time/taxi throughput 0.333668 0.295879 1.12772 bytes/ns
parquet_rs-zstd compress time/taxi 1.77948e+09 1.81144e+09 0.982356 ns
parquet_rs-zstd compress time/taxi throughput 0.264577 0.259909 1.01796 bytes/ns
decompress time/taxi 3.9286e+08 3.7288e+08 1.05358 ns
decompress time/taxi throughput 1.19842 1.26263 0.949142 bytes/ns
parquet_rs-zstd decompress time/taxi 3.08558e+08 3.09274e+08 0.997685 ns
parquet_rs-zstd decompress time/taxi throughput 1.52584 1.52231 1.00232 bytes/ns
compress time/AirlineSentiment 289604 288290 1.00456 ns
compress time/AirlineSentiment throughput 0.0070441 0.00707621 0.995462 bytes/ns
parquet_rs-zstd compress time/AirlineSentiment 56573.8 55323.5 1.0226 ns
parquet_rs-zstd compress time/AirlineSentiment throughput 0.0360591 0.036874 0.977899 bytes/ns
decompress time/AirlineSentiment 188053 189176 0.99406 ns
decompress time/AirlineSentiment throughput 0.010848 0.0107836 1.00598 bytes/ns
parquet_rs-zstd decompress time/AirlineSentiment 30602.6 31384.1 0.975099 ns
parquet_rs-zstd decompress time/AirlineSentiment throughput 0.066661 0.0650011 1.02554 bytes/ns
compress time/Arade 2.74469e+09 2.73041e+09 1.00523 ns
compress time/Arade throughput 0.286746 0.288245 0.994799 bytes/ns
parquet_rs-zstd compress time/Arade 2.96725e+09 3.05469e+09 0.971374 ns
parquet_rs-zstd compress time/Arade throughput 0.265239 0.257646 1.02947 bytes/ns
decompress time/Arade 7.3804e+08 7.36221e+08 1.00247 ns
decompress time/Arade throughput 1.06638 1.06901 0.997535 bytes/ns
parquet_rs-zstd decompress time/Arade 6.65969e+08 6.75512e+08 0.985872 ns
parquet_rs-zstd decompress time/Arade throughput 1.18178 1.16508 1.01433 bytes/ns
compress time/Bimbo 1.10451e+10 1.18008e+10 0.935958 ns
compress time/Bimbo throughput 0.644753 0.603462 1.06842 bytes/ns
parquet_rs-zstd compress time/Bimbo 2.11682e+10 2.23008e+10 0.949214 ns
parquet_rs-zstd compress time/Bimbo throughput 0.336417 0.319332 1.0535 bytes/ns
decompress time/Bimbo 4.42028e+09 4.64748e+09 0.951113 ns
decompress time/Bimbo throughput 1.61106 1.5323 1.0514 bytes/ns
parquet_rs-zstd decompress time/Bimbo 3.41097e+09 3.98815e+09 0.855276 ns
parquet_rs-zstd decompress time/Bimbo throughput 2.08778 1.78563 1.16921 bytes/ns
compress time/CMSprovider 1.26872e+10 1.26459e+10 1.00327 ns
compress time/CMSprovider throughput 0.405854 0.407181 0.996742 bytes/ns
parquet_rs-zstd compress time/CMSprovider 1.85001e+10 1.8699e+10 0.989362 ns
parquet_rs-zstd compress time/CMSprovider throughput 0.278331 0.27537 1.01075 bytes/ns
decompress time/CMSprovider 4.7986e+09 4.62022e+09 1.03861 ns
decompress time/CMSprovider throughput 1.07305 1.11448 0.962826 bytes/ns
parquet_rs-zstd decompress time/CMSprovider 5.44584e+09 5.28598e+09 1.03024 ns
parquet_rs-zstd decompress time/CMSprovider throughput 0.945522 0.974116 0.970646 bytes/ns
compress time/Euro2016 2.43452e+09 2.08768e+09 1.16614 ns
compress time/Euro2016 throughput 0.161533 0.188369 0.857532 bytes/ns
parquet_rs-zstd compress time/Euro2016 1.54087e+09 1.5647e+09 0.98477 ns
parquet_rs-zstd compress time/Euro2016 throughput 0.255217 0.25133 1.01547 bytes/ns
decompress time/Euro2016 2.78091e+08 2.83796e+08 0.979896 ns
decompress time/Euro2016 throughput 1.41412 1.38569 1.02052 bytes/ns
parquet_rs-zstd decompress time/Euro2016 4.84884e+08 5.02367e+08 0.965199 ns
parquet_rs-zstd decompress time/Euro2016 throughput 0.811028 0.782803 1.03606 bytes/ns
compress time/Food 1.03251e+09 1.04563e+09 0.987447 ns
compress time/Food throughput 0.322244 0.318199 1.01271 bytes/ns
parquet_rs-zstd compress time/Food 1.05962e+09 1.0725e+09 0.987993 ns
parquet_rs-zstd compress time/Food throughput 0.313998 0.310228 1.01215 bytes/ns
decompress time/Food 1.79929e+08 1.84156e+08 0.977047 ns
decompress time/Food throughput 1.84917 1.80673 1.02349 bytes/ns
parquet_rs-zstd decompress time/Food 2.21281e+08 2.26448e+08 0.977186 ns
parquet_rs-zstd decompress time/Food throughput 1.50361 1.4693 1.02335 bytes/ns
compress time/HashTags 2.79918e+09 2.50168e+09 1.11892 ns
compress time/HashTags throughput 0.287405 0.321584 0.893717 bytes/ns
parquet_rs-zstd compress time/HashTags 2.45431e+09 2.48444e+09 0.987872 ns
parquet_rs-zstd compress time/HashTags throughput 0.32779 0.323815 1.01228 bytes/ns
decompress time/HashTags 4.47872e+08 4.53141e+08 0.988373 ns
decompress time/HashTags throughput 1.79627 1.77538 1.01176 bytes/ns
parquet_rs-zstd decompress time/HashTags 7.82401e+08 8.05723e+08 0.971054 ns
parquet_rs-zstd decompress time/HashTags throughput 1.02824 0.998481 1.02981 bytes/ns
compress time/TPC-H l_comment chunked without fsst 2.99146e+09 3.08176e+09 0.970701 ns
compress time/TPC-H l_comment chunked without fsst throughput 0.0833032 0.0808625 1.03018 bytes/ns
parquet_rs-zstd compress time/TPC-H l_comment chunked without fsst 8.95806e+08 9.16784e+08 0.977118 ns
parquet_rs-zstd compress time/TPC-H l_comment chunked without fsst throughput 0.278184 0.271818 1.02342 bytes/ns
decompress time/TPC-H l_comment chunked without fsst 5.30299e+07 5.55432e+07 0.95475 ns
decompress time/TPC-H l_comment chunked without fsst throughput 4.69921 4.48657 1.04739 bytes/ns
parquet_rs-zstd decompress time/TPC-H l_comment chunked without fsst 2.50606e+08 2.49905e+08 1.0028 ns
parquet_rs-zstd decompress time/TPC-H l_comment chunked without fsst throughput 0.994384 0.997173 0.997204 bytes/ns
compress time/TPC-H l_comment chunked 1.62685e+09 1.02256e+09 1.59095 ns
compress time/TPC-H l_comment chunked throughput 0.153179 0.243701 0.628554 bytes/ns
parquet_rs-zstd compress time/TPC-H l_comment chunked 8.99401e+08 9.16079e+08 0.981794 ns
parquet_rs-zstd compress time/TPC-H l_comment chunked throughput 0.277072 0.272027 1.01854 bytes/ns
decompress time/TPC-H l_comment chunked 1.09384e+08 1.01917e+08 1.07327 ns
decompress time/TPC-H l_comment chunked throughput 2.27819 2.44511 0.931731 bytes/ns
parquet_rs-zstd decompress time/TPC-H l_comment chunked 2.50231e+08 2.48296e+08 1.00779 ns
parquet_rs-zstd decompress time/TPC-H l_comment chunked throughput 0.995874 1.00363 0.992268 bytes/ns
compress time/TPC-H l_comment canonical 1.61883e+09 1.02606e+09 1.57771 ns
compress time/TPC-H l_comment canonical throughput 0.153937 0.242868 0.633831 bytes/ns
parquet_rs-zstd compress time/TPC-H l_comment canonical 9.00428e+08 9.15215e+08 0.983843 ns
parquet_rs-zstd compress time/TPC-H l_comment canonical throughput 0.276755 0.272283 1.01642 bytes/ns
decompress time/TPC-H l_comment canonical 1.09852e+08 1.01422e+08 1.08312 ns
decompress time/TPC-H l_comment canonical throughput 2.2685 2.45705 0.923261 bytes/ns
parquet_rs-zstd decompress time/TPC-H l_comment canonical 2.50375e+08 2.50957e+08 0.997683 ns
parquet_rs-zstd decompress time/TPC-H l_comment canonical throughput 0.995298 0.992992 1.00232 bytes/ns

@lwwmanning lwwmanning added the benchmark Run benchmarks on this branch label Jan 13, 2025
@github-actions github-actions bot removed the benchmark Run benchmarks on this branch label Jan 13, 2025
@lwwmanning lwwmanning mentioned this pull request Jan 21, 2025
Merged
@lwwmanning lwwmanning added the benchmark Run benchmarks on this branch label Jan 21, 2025
@github-actions github-actions bot removed the benchmark Run benchmarks on this branch label Jan 21, 2025
@gatesn gatesn mentioned this pull request Jan 23, 2025
Open
@lwwmanning lwwmanning marked this pull request as draft January 23, 2025 16:50
@lwwmanning lwwmanning mentioned this pull request Jan 30, 2025
Merged
@lwwmanning lwwmanning added the benchmark Run benchmarks on this branch label Jan 30, 2025
@github-actions github-actions bot removed the benchmark Run benchmarks on this branch label Jan 30, 2025
@lwwmanning lwwmanning mentioned this pull request Jan 30, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@lwwmanning