Open
Description
The first version of writing into deltalake did not implement overwriting an existing table. Currently, this is raising a ValueError
(should actually raise a NotImplementedError
):
import pandas as pd
import numpy as np
import dask.dataframe as dd
from dask_deltatable.write import to_deltalake
if __name__ == "__main__":
df = pd.DataFrame({
"i1": np.random.randint(1, 10000, size=100),
"f1": np.random.random(100),
"s1": np.random.choice(["Apple", "Banana", "Watermelon", "Mango"], size=100),
})
ddf = dd.from_pandas(df, npartitions=10)
to_deltalake("t1_data", ddf, mode="overwrite").compute()
Raises:
Traceback (most recent call last):
File "/Users/jbennet/src/dask-deltatable/t1.py", line 14, in <module>
to_deltalake("t1_data", ddf, mode="overwrite").compute()
File "/Users/jbennet/src/dask-deltatable/dask_deltatable/write.py", line 82, in to_deltalake
raise ValueError(
ValueError: Schema of data does not match table schema
Table schema:
None
Data Schema:
i1: int64
f1: double
s1: string
Original PR: #29.
Metadata
Metadata
Assignees
Labels
No labels