You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have an issue reading delta tables using deltatorch when the lib wawnts to build the PyArrow dataset out of the delta table.
As latest Databricks runtimes create delta tables with high feature level (3, 7) and those are different from the ones I see into the code base:
the execution results in error: DeltaProtocolError: The table's minimum reader version is 3 but deltalake only supports up to version 1.
coming from the code base here:
if self.protocol().min_reader_version > MAX_SUPPORTED_READER_VERSION:
raise DeltaProtocolError(
f"The table's minimum reader version is {self.protocol().min_reader_version} "
f"but deltalake only supports up to version {MAX_SUPPORTED_READER_VERSION}."
)
What can be done to overcome this error? Why is it necessary that deltatorch keeps the protocol feature level to (1,2) ?
Document the table features / protocol version that are supported by this project for each release.
If a Delta table has a table feature enabled that deltatorch does not support (like deletion vectors), then deltatorch should error out.
The text was updated successfully, but these errors were encountered: