Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dependency issues when running new catalog workflow #2

Open
akadlof opened this issue Jan 28, 2025 · 0 comments · Fixed by #4
Open

Dependency issues when running new catalog workflow #2

akadlof opened this issue Jan 28, 2025 · 0 comments · Fixed by #4

Comments

@akadlof
Copy link
Collaborator

akadlof commented Jan 28, 2025

Steps to reproduce

  1. Clone repo, enter venv, and run pip install -e .
  2. Update .\workflows\new.py to run on subset of provided sample data:
from stormhub.logger import initialize_logger
from stormhub.met.storm_catalog import new_catalog, new_collection

if __name__ == "__main__":
    initialize_logger()

    # Catalog Args
    root_dir = "catalogs/example-input-data"
    config_file = f"{root_dir}/config.json"
    catalog_id = "indian-creek"
    local_directory = f"{root_dir}"

    storm_catalog = new_catalog(
        catalog_id,
        config_file,
        local_directory=local_directory,
        catalog_description="Indian Creek Catalog",
    )

    # All Collection Args
    start_date = "2023-8-01"
    end_date = "2023-8-08"
    top_n_events = 440

    # Collection Args
    storm_duration_hours = 48
    min_precip_threshold = 2.5
    storm_collection = new_collection(
        storm_catalog,
        start_date,
        end_date,
        storm_duration_hours,
        min_precip_threshold,
        top_n_events,
        check_every_n_hours=6,
    )
  1. Run python .\workflows\new.py

Expected behavior

The new catalog workflow runs without errors.

Current behavior

File "stormhub\stormhub\met\aorc\aorc.py", line 8, in <module> import s3fs ModuleNotFoundError: No module named 's3fs'

After pip installing s3fs (version 2024.12.0), I get:
File "stormhub\venv\Lib\site-packages\xarray\backends\plugins.py", line 202, in get_backend raise ValueError( ValueError: unrecognized engine 'zarr' must be one of your download engines: ['rasterio', 'store'].

After pip install of zarr, I get:
File "stormhub\venv\Lib\site-packages\zarr\storage\_common.py", line 316, in make_store_path raise TypeError(msg) TypeError: Unsupported type for store_like: 'FSMap'

Which is resolved by downgrading to version 2.18.4 (link to more info)

After downgrade of zarr, I get:
File "stormhub\venv\Lib\site-packages\xarray\namedarray\parallelcompat.py", line 110, in guess_chunkmanager raise ValueError( ValueError: unrecognized chunk manager dask - must be one of: []

Which is resolved by pip installing dask (version 2025.1.0)

At this point the workflow runs without errors.

Environment

  • Windows 11
  • Python 3.11
@sray014 sray014 linked a pull request Jan 31, 2025 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant