You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When switching a Python writer to use the Rust engine (while testing #2486) I've discovered that the Rust engine doesn't errors on the following code only for the Rust engine:
…rect behavior
Uses of mode='append' and overwrite_schema=True lead to inconsistent
behavior between Rust and PyArrow engines for write_deltalake. In the
PyArrow case the parameter is quietly omitted so users may experience
unexpected behavior since schemas will not actually be overridden.
Users of this parameter set most likely want schema_mode='merge' which
would allow for schema evolution on appends to a Delta Table
Fixesdelta-io#2553
rtyler
added a commit
to rtyler/delta-rs
that referenced
this issue
May 29, 2024
…rect behavior
Uses of mode='append' and overwrite_schema=True lead to inconsistent
behavior between Rust and PyArrow engines for write_deltalake. In the
PyArrow case the parameter is quietly omitted so users may experience
unexpected behavior since schemas will not actually be overridden.
Users of this parameter set most likely want schema_mode='merge' which
would allow for schema evolution on appends to a Delta Table
Fixesdelta-io#2553
…rect behavior
Uses of mode='append' and overwrite_schema=True lead to inconsistent
behavior between Rust and PyArrow engines for write_deltalake. In the
PyArrow case the parameter is quietly omitted so users may experience
unexpected behavior since schemas will not actually be overridden.
Users of this parameter set most likely want schema_mode='merge' which
would allow for schema evolution on appends to a Delta Table
Fixes#2553
Environment
Delta-rs version: current
Binding: python
Bug
What happened:
When switching a Python writer to use the Rust engine (while testing #2486) I've discovered that the Rust engine doesn't errors on the following code only for the Rust engine:
Fails with:
What you expected to happen:
I would expect the engine change to be seamless
More details:
🤕
The text was updated successfully, but these errors were encountered: