[FEAT] connect: support basic column operations

[andrewgazelka]

Works:

Column.alias(*alias, **kwargs)
Column.cast(dataType)
Column.isNotNull()
Column.isNull()
Column.name(*alias, **kwargs) (= name)

[andrewgazelka]

• [FEAT] connect: support basic column operations #3362 : 2 dependent PRs (#3350 , #3363 ) 👈 (View in Graphite)
• [FEAT] connect: to_daft_* use ref instead of value #3355 : 2 other dependent PRs (#3357 , #3360 )
• [FEAT] connect: add alias support #3342 : 6 other dependent PRs (#3344 , #3345 , #3348 and 3 others)
• [FEAT] consolidate Spark session fixture into conftest.py #3341 : 3 other dependent PRs (#3309 , #3326 , #3343 )
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[codspeed-hq]

CodSpeed Performance Report

Merging #3362 will improve performances by 54.69%

_{Comparing andrew/connect-column-operations (a0165a6) with main (83470e0)}

Summary

⚡ 2 improvements
✅ 15 untouched benchmarks

Benchmarks breakdown

	Benchmark	main	andrew/connect-column-operations	Change
⚡	test_iter_rows_first_row[100 Small Files]	283.1 ms	238.6 ms	+18.65%
⚡	test_show[100 Small Files]	25.8 ms	16.7 ms	+54.69%

[graphite-app]

Graphite Automations

"Notify author when CI fails" took an action on this PR • (11/21/24)

1 teammate was notified to this PR based on Andrew Gazelka's automation.

[codecov]

Codecov Report

Attention: Patch coverage is 25.44379% with 126 lines in your changes missing coverage. Please review.

Project coverage is 77.33%. Comparing base (83470e0) to head (a0165a6).
Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
src/daft-connect/src/translation/datatype.rs	7.96%	104 Missing ⚠️
src/daft-connect/src/translation/expr.rs	50.00%	18 Missing ⚠️
...onnect/src/translation/expr/unresolved_function.rs	80.00%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3362      +/-   ##
==========================================
- Coverage   77.39%   77.33%   -0.06%     
==========================================
  Files         696      696              
  Lines       84549    84716     +167     
==========================================
+ Hits        65438    65517      +79     
- Misses      19111    19199      +88     

Files with missing lines	Coverage Δ
src/daft-connect/src/lib.rs	64.21% <ø> (ø)
...onnect/src/translation/expr/unresolved_function.rs	64.70% <80.00%> (+64.70%)	⬆️
src/daft-connect/src/translation/expr.rs	57.14% <50.00%> (+1.00%)	⬆️
src/daft-connect/src/translation/datatype.rs	10.40% <7.96%> (+2.99%)	⬆️

... and 4 files with indirect coverage changes

[jaychia]

PR LGTM, added some comments but mostly questions.

Please do address the comment on # Test cast though.

[andrewgazelka]

Merge activity

• Dec 3, 9:32 PM EST: A user merged this pull request with Graphite.