Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows paths: spaces are parsed to %20 (reading delta table) #1391

Closed
ABChristian opened this issue May 25, 2023 · 6 comments
Closed

Windows paths: spaces are parsed to %20 (reading delta table) #1391

ABChristian opened this issue May 25, 2023 · 6 comments
Labels
bug Something isn't working

Comments

@ABChristian
Copy link

ABChristian commented May 25, 2023

Environment

Delta-rs version:
Python deltalake 0.9.0

Environment:

  • OS: Windows 10 64bit
  • Python: 3.11.3

Bug

What happened:
Reading a delta table inside a folder which contains spaces thorws an error.
A new folder containing "%20" instead of spaces is created.

This folder/delta table was written using deltalake, and created in the expected location.

What you expected to happen:
The table should be created and read in the same folder.

How to reproduce it:
`import deltalake as dl

dl.write_deltalake(table_or_uri="./deltalake/data",data=DATAFRAME, mode="append")
data = dl.DeltaTable(table_uri="./deltalake/data")
print(data.to_pandas())`

More details:
File "(...)\Lib\site-packages\deltalake\table.py", line 442, in to_pandas
return self.to_pyarrow_table(
^^^^^^^^^^^^^^^^^^^^^^
File "(...)\Lib\site-packages\deltalake\table.py", line 426, in to_pyarrow_table
).to_table(columns=columns)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "pyarrow_dataset.pyx", line 546, in pyarrow._dataset.Dataset.to_table
File "pyarrow_dataset.pyx", line 3449, in pyarrow._dataset.Scanner.to_table
File "pyarrow\error.pxi", line 144, in pyarrow.lib.pyarrow_internal_check_status
File "pyarrow_fs.pyx", line 1551, in pyarrow._fs._cb_open_input_file
File "(...)\Lib\site-packages\deltalake\fs.py", line 22, in open_input_file
return pa.PythonFile(DeltaFileSystemHandler.open_input_file(self, path))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
deltalake.PyDeltaTableError: Object at location C:\Folders\Prefix%20-%20Suffix\deltalake\data\0-8162c431-b69c-4931-8618-75de0202cbc6-0.parquet not found: (...) (os error 2)

@ABChristian ABChristian added the bug Something isn't working label May 25, 2023
@djouallah
Copy link

same issue here

@Phil-T1
Copy link

Phil-T1 commented Sep 25, 2023

Using 0.10.2, I'm having this same issue where Windows paths cannot be decoded due to their spaceyness.

Writing only succeeds where paths which have no space characters. :(

@JvdH-NL
Copy link

JvdH-NL commented Apr 17, 2024

Using 0.16.4 I am having same issue. Running Python code from a path that contains spaces gives me 'object at location <FILL PATH WITH SPACES that have been replaced with %20 .... parquet file> not found ....
I have some sample data, using code, fails on last statement. And path is on corporate OneDrive containing spaces.
delta_table_path = 'deltaTable/'
dt = DeltaTable(delta_table_path)

Read Data from Delta table

dt.to_pandas()

@JvdH-NL
Copy link

JvdH-NL commented Apr 17, 2024

Quickstart on Homepage results in error in reading (writing part goes ok), same issue as my previous post.

@ion-elgreco
Copy link
Collaborator

Is this still an issue with 0.22.3?

@ABChristian
Copy link
Author

It's working!
Tested using deltalake 0.22.3, Python 3.12.8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

5 participants