Errors opening S3Path with rasterio and rioxarray

[cas--]

I'm not sure if this is a upath or s3fs issue but when trying to pass S3Path to rioxarray I am encountering errors.

For local filesystem there are no issues, included cloudpathlib for comparison:

❯ uv run -s rasterio_upath_error.py test.tif 
Open using cloudpathlib AnyPath for path: test.tif
Open using universal-pathlib UPath for path: tests.tif

For S3 paths opens fine with cloudpathlib but fails with universal-pathlib

uv run -s rasterio_upath_error.py s3://bucket/test.tif
Open using cloudpathlib AnyPath for path: s3://bucket/test.tif
Open using universal-pathlib UPath for path: s3://bucket/test.tif
Traceback (most recent call last):
  File ".cache/uv/environments-v2/rasterio-upath-error-423a0df2cfd1344e/lib/python3.12/site-packages/xarray/backends/file_manager.py", line 211, in _acquire_with_cache_info
    file = self._cache[self._key]
           ~~~~~~~~~~~^^^^^^^^^^^
  File ".cache/uv/environments-v2/rasterio-upath-error-423a0df2cfd1344e/lib/python3.12/site-packages/xarray/backends/lru_cache.py", line 56, in __getitem__
    value = self._cache[key]
            ~~~~~~~~~~~^^^^^
KeyError: [<function open at 0x76955c36b600>, (S3Path('s3://bucket/test.tif'),), 'r', (('sharing', False),), '1535b53f-793d-44de-ac73-393caa242dd5']

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "rasterio/_vsiopener.pyx", line 381, in rasterio._vsiopener._opener_registration
  File "rasterio/_vsiopener.pyx", line 594, in rasterio._vsiopener._FileContainer.size
  File ".cache/uv/environments-v2/rasterio-upath-error-423a0df2cfd1344e/lib/python3.12/site-packages/fsspec/spec.py", line 1310, in open
    f = self._open(
        ^^^^^^^^^^^
  File ".cache/uv/environments-v2/rasterio-upath-error-423a0df2cfd1344e/lib/python3.12/site-packages/s3fs/core.py", line 720, in _open
    return S3File(
           ^^^^^^^
  File ".cache/uv/environments-v2/rasterio-upath-error-423a0df2cfd1344e/lib/python3.12/site-packages/s3fs/core.py", line 2232, in __init__
    raise ValueError("Attempt to open non key-like path: %s" % path)
ValueError: Attempt to open non key-like path: test

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "rasterio_upath_error.py", line 30, in <module>
    main()
  File "rasterio_upath_error.py", line 25, in main
    with rxr.open_rasterio(UPath(path)) as ds:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".cache/uv/environments-v2/rasterio-upath-error-423a0df2cfd1344e/lib/python3.12/site-packages/rioxarray/_io.py", line 1135, in open_rasterio
    riods = manager.acquire()
            ^^^^^^^^^^^^^^^^^
  File ".cache/uv/environments-v2/rasterio-upath-error-423a0df2cfd1344e/lib/python3.12/site-packages/xarray/backends/file_manager.py", line 193, in acquire
    file, _ = self._acquire_with_cache_info(needs_lock)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".cache/uv/environments-v2/rasterio-upath-error-423a0df2cfd1344e/lib/python3.12/site-packages/xarray/backends/file_manager.py", line 217, in _acquire_with_cache_info
    file = self._opener(*self._args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".cache/uv/environments-v2/rasterio-upath-error-423a0df2cfd1344e/lib/python3.12/site-packages/rasterio/env.py", line 463, in wrapper
    return f(*args, **kwds)
           ^^^^^^^^^^^^^^^^
  File ".cache/uv/environments-v2/rasterio-upath-error-423a0df2cfd1344e/lib/python3.12/site-packages/rasterio/__init__.py", line 350, in open
    registered_vsi_path = stack.enter_context(vsi_path_ctx)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".local/share/uv/python/cpython-3.12.6-linux-x86_64-gnu/lib/python3.12/contextlib.py", line 526, in enter_context
    result = _enter(cm)
             ^^^^^^^^^^
  File ".local/share/uv/python/cpython-3.12.6-linux-x86_64-gnu/lib/python3.12/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "rasterio/_vsiopener.pyx", line 383, in _opener_registration
rasterio.errors.OpenerRegistrationError: Opener is invalid.

Script used to test

#!/usr/bin/env -S uv run -s
# /// script
# requires-python = ">=3.12"
# dependencies = [
#     "cloudpathlib[s3]",
#     "fsspec[s3]",
#     "rioxarray",
#     "universal-pathlib",
# ]
# ///
import sys
import rioxarray as rxr
from xarray import DataArray
from upath import UPath
from cloudpathlib import AnyPath


def main() -> None:
    path = sys.argv[1]
    print(f"Open using cloudpathlib AnyPath for path: {path}")
    with rxr.open_rasterio(AnyPath(path)) as ds:
        assert isinstance(ds, DataArray)

    print(f"Open using universal-pathlib UPath for path: {path}")
    with rxr.open_rasterio(UPath(path)) as ds:
        assert isinstance(ds, DataArray)


if __name__ == "__main__":
    main()

[ap--]

My guess is that this is related to the fact that UPath currently incorrectly pretends to be a local path... I.e os.fspath(UPath("s3://bucket/file")) returns s3://bucket/file.

xarray then likely tries to use a file reader for local files and fails.

This issue was fixed in #366 and will be available in the next release of universal-pathlib, which should happen soon. I just need to finish #346 before I can make the release.

[cas--]

I've tried with latest from github but although it resolves the cache KeyError the others errors still occur:

❯ ./rasterio_upath_latest_error.py
...
ValueError: Attempt to open non key-like path: test
...
rasterio.errors.OpenerRegistrationError: Opener is invalid.

Here is an updated script with example:

#!/usr/bin/env -S uv run --no-project
# /// script
# requires-python = ">=3.12"
# dependencies = [
#     "fsspec[s3]",
#     "rioxarray",
#     "universal-pathlib",
# ]
#
# [tool.uv.sources]
# universal-pathlib = { git = "https://github.com/fsspec/universal_pathlib" }
# ///
import rioxarray as rxr
from upath import UPath

path = UPath(
    "s3://sentinel-cogs/sentinel-s2-l2a-cogs/39/G/VJ/2025/5/S2B_39GVJ_20250519_0_L2A/AOT.tif",
    anon=True,
)
assert path.is_file()
print(f"Opening path: {path}")
with rxr.open_rasterio(path) as ds:
    print(ds)