Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failing to read files using DirFileSystem #461

Closed
rudyryk opened this issue Nov 27, 2024 · 0 comments · Fixed by #464
Closed

Failing to read files using DirFileSystem #461

rudyryk opened this issue Nov 27, 2024 · 0 comments · Fixed by #464

Comments

@rudyryk
Copy link

rudyryk commented Nov 27, 2024

We get Permission Error: File system LocalFileSystem has been disabled by configuration exception when trying to read via st_read().

Indeed we disable LocalFileSystem but we provide registered dir:// file system which can be normally used by read_csv().

Here's an example to reproduce the issue:

import duckdb
from fsspec import filesystem

duckdb.install_extension('spatial')
duckdb.load_extension('spatial')
duckdb.register_filesystem(filesystem('dir', path='/tmp/'))

# This works
csv_result = duckdb.sql("SELECT * FROM read_csv('dir://test.csv', null_padding = true) LIMIT 2")
print(csv_result)

# This works
xls_result = duckdb.sql("SELECT * FROM st_read('dir://test.xlsx') LIMIT 2")
print(xls_result)

# Disable local file system
duckdb.execute("SET disabled_filesystems = 'LocalFileSystem'")

# This still works
csv_result2 = duckdb.sql("SELECT * FROM read_csv('dir://test.csv', null_padding = true) LIMIT 3")
print(csv_result2)

# This now fails!
xls_result2 = duckdb.sql("SELECT * FROM st_read('dir://test.xlsx') LIMIT 3")
print(xls_result2)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant