-
-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(python): Enable view arrow export in write_delta
#20092
feat(python): Enable view arrow export in write_delta
#20092
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #20092 +/- ##
==========================================
- Coverage 79.53% 79.53% -0.01%
==========================================
Files 1563 1563
Lines 217124 217119 -5
Branches 2464 2465 +1
==========================================
- Hits 172680 172675 -5
+ Misses 43884 43883 -1
- Partials 560 561 +1 ☔ View full report in Codecov by Sentry. |
Nice, can you ping once it is released? |
@@ -4316,7 +4321,6 @@ def write_delta( | |||
schema=schema, | |||
mode=mode, | |||
storage_options=storage_options, | |||
large_dtypes=True, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If version <= 0.22.3 we should still pass large_dtypes=True
, I think.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually, since v0.19 that parameter doesn't do anything when the Rust engine is used for writing, which is the default. I'll change the minimum in pyproject.toml to >=0.19
write_delta
write_delta
Btw, we can merge this already right? Why do we need to wait on the release? ^^ |
We don't actually since we do version check :P, let's merge it then! |
In next deltalake 0.22.3 release the view types will be fully supported.
Only merge once 0.22.3 is released
cc @ritchie46