Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace FilePointer with universal pathlib #387

Merged
merged 7 commits into from
Sep 13, 2024
Merged

Conversation

delucchi-cmu
Copy link
Contributor

See astronomy-commons/hipscat#336

Solution Description

Use fsspec's project, universal-pathlib as a path provider. In this way, everywhere we currently pass in a FilePointer we can use a Path instead, and when we need to create remote connections, use UPath.

Copy link

codecov bot commented Sep 12, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.73%. Comparing base (c61c1a7) to head (2d453ac).
Report is 2 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #387   +/-   ##
=======================================
  Coverage   99.73%   99.73%           
=======================================
  Files          26       26           
  Lines        1507     1520   +13     
=======================================
+ Hits         1503     1516   +13     
  Misses          4        4           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link

github-actions bot commented Sep 12, 2024

Before [827aac0] After [2cbb914] Ratio Benchmark (Parameter)
failed failed n/a benchmarks.BinningSuite.time_read_histogram

Click here to view all benchmarks.

Copy link
Contributor

@hombit hombit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love how it makes everything much simpler!

There is an inconsistency in *Arguments' parameters type-hints: some of them are UPath, some are str (the same for List[*]). Only for these "end-user" interfaces, may I ask to use something more broad, like str | Path | UPath?

Do we have a policy that requires any imported package to be an explicit dependency? (it is a good one!) Should we add universal_pathlib to pyproject.toml?

@delucchi-cmu
Copy link
Contributor Author

I love how it makes everything much simpler!

There is an inconsistency in *Arguments' parameters type-hints: some of them are UPath, some are str (the same for List[*]). Only for these "end-user" interfaces, may I ask to use something more broad, like str | Path | UPath?

Do we have a policy that requires any imported package to be an explicit dependency? (it is a good one!) Should we add universal_pathlib to pyproject.toml?

Added a lot more type hints, and direct dependency on universal_pathlib.

Copy link
Contributor

@hombit hombit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@delucchi-cmu delucchi-cmu merged commit 829fe47 into main Sep 13, 2024
12 checks passed
@delucchi-cmu delucchi-cmu deleted the delucchi/upath branch September 13, 2024 16:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants