feat(connect): add parquet support

[andrewgazelka]

No description provided.

[andrewgazelka]

• feat(connect): read/write → csv, write → json #3361
• feat(connect): add parquet support #3360 👈 (View in Graphite)
• [FEAT] connect: to_daft_* use ref instead of value #3355 : 2 other dependent PRs (#3357 , #3362 )
• [FEAT] connect: add alias support #3342 : 6 other dependent PRs (#3344 , #3345 , #3348 and 3 others)
• [FEAT] consolidate Spark session fixture into conftest.py #3341 : 3 other dependent PRs (#3309 , #3326 , #3343 )
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[codspeed-hq]

CodSpeed Performance Report

Merging #3360 will degrade performances by 39.79%

_{Comparing andrew/connect-parquet3 (4a1d365) with main (f23ee37)}

Summary

⚡ 1 improvements
❌ 1 regressions
✅ 15 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

	Benchmark	main	andrew/connect-parquet3	Change
❌	test_iter_rows_first_row[100 Small Files]	189.2 ms	314.3 ms	-39.79%
⚡	test_show[100 Small Files]	23.8 ms	16.1 ms	+47.64%

[codecov]

Codecov Report

Attention: Patch coverage is 80.34682% with 34 lines in your changes missing coverage. Please review.

Project coverage is 77.68%. Comparing base (e5ea73f) to head (4a1d365).
Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
src/daft-connect/src/op/execute/write.rs	78.31%	18 Missing ⚠️
src/daft-connect/src/translation/logical_plan.rs	76.19%	5 Missing ⚠️
.../daft-connect/src/translation/logical_plan/read.rs	73.68%	5 Missing ⚠️
...t/src/translation/logical_plan/read/data_source.rs	84.37%	5 Missing ⚠️
...daft-connect/src/translation/logical_plan/to_df.rs	75.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3360      +/-   ##
==========================================
+ Coverage   76.11%   77.68%   +1.56%     
==========================================
  Files         710      713       +3     
  Lines       89196    87091    -2105     
==========================================
- Hits        67894    67657     -237     
+ Misses      21302    19434    -1868     

Files with missing lines	Coverage Δ
src/daft-connect/src/lib.rs	68.57% <100.00%> (+2.22%)	⬆️
src/daft-connect/src/op/execute.rs	80.95% <ø> (ø)
src/daft-connect/src/op/execute/root.rs	95.65% <100.00%> (+0.19%)	⬆️
...-connect/src/translation/logical_plan/aggregate.rs	81.81% <100.00%> (ø)
...ft-connect/src/translation/logical_plan/project.rs	88.88% <100.00%> (ø)
...nnect/src/translation/logical_plan/with_columns.rs	94.73% <100.00%> (ø)
src/daft-connect/src/translation/schema.rs	100.00% <ø> (ø)
...daft-connect/src/translation/logical_plan/to_df.rs	90.00% <75.00%> (-4.74%)	⬇️
src/daft-connect/src/translation/logical_plan.rs	69.23% <76.19%> (-3.94%)	⬇️
.../daft-connect/src/translation/logical_plan/read.rs	73.68% <73.68%> (ø)
... and 2 more

... and 31 files with indirect coverage changes