Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for S3 in HubModel #2378

Merged
merged 32 commits into from
Jan 19, 2024
Merged

Add support for S3 in HubModel #2378

merged 32 commits into from
Jan 19, 2024

Conversation

martinkim0
Copy link
Contributor

@martinkim0 martinkim0 commented Jan 5, 2024

  • Add pull_from_s3 method to initialize a HubModel
  • Allow different ways of passing in credentials
  • Add public test files to s3 bucket
  • Add tests
  • Add push to s3 method
  • Authenticate AWS in GitHub actions

Copy link

codecov bot commented Jan 5, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (6078041) 89.17% compared to head (3f44406) 89.41%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2378      +/-   ##
==========================================
+ Coverage   89.17%   89.41%   +0.23%     
==========================================
  Files         153      153              
  Lines       12518    12564      +46     
==========================================
+ Hits        11163    11234      +71     
+ Misses       1355     1330      -25     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@canergen
Copy link
Member

canergen commented Jan 6, 2024

Working. It would be good to check cache_dir exists and otherwise create it.

@martinkim0 martinkim0 changed the title Add pull_from_s3 for HubModel Add support for S3 in HubModel Jan 10, 2024
Copy link
Member

@canergen canergen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@martinkim0 martinkim0 added the on-merge: backport to 1.1.x on-merge: backport to 1.1.x label Jan 11, 2024
@martinkim0 martinkim0 merged commit a03401d into main Jan 19, 2024
8 of 9 checks passed
@martinkim0 martinkim0 deleted the hub_s3 branch January 19, 2024 01:18
meeseeksmachine pushed a commit to meeseeksmachine/scvi-tools that referenced this pull request Jan 19, 2024
martinkim0 added a commit that referenced this pull request Jan 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
on-merge: backport to 1.1.x on-merge: backport to 1.1.x
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants