Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds support for the new Terra DRSHub #41

Merged
merged 20 commits into from
Sep 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
89031e1
Adds functionality for new TDR DRS GCP Requester Pays Feature
matthewpeterkort Sep 18, 2023
7ce2bf2
flake8
matthewpeterkort Sep 25, 2023
6ba344d
Updates and fixes tests, cleans up fixtures
matthewpeterkort Sep 25, 2023
5910ef9
adds small change to aiohttp lib to avoid snyk vulnerability
matthewpeterkort Sep 27, 2023
f6cf7bb
changes to reflect doc. Testing
matthewpeterkort Oct 3, 2023
9db9788
Adds invalid project error handling and tested with mixed pays data
matthewpeterkort Oct 5, 2023
86bb86f
adds tests for new functionality
matthewpeterkort Oct 5, 2023
9e82a05
Add macOS package signing to Github Actions workflow
lbeckman314 Oct 16, 2023
ebd2117
Merge branch 'main' into feature/testing-dev
lbeckman314 Nov 7, 2023
ae59846
Adds greater functionality for checking args
matthewpeterkort Nov 8, 2023
3698c81
flake8
matthewpeterkort Nov 8, 2023
a39534a
Addresses the DRS URI identification issue
matthewpeterkort Nov 9, 2023
8651000
adds support for uppercase uri prefixess
matthewpeterkort Nov 9, 2023
20ddbbc
Adds verbose logging to the drs_downloader.log file, and seperate log…
matthewpeterkort Nov 9, 2023
4b1671c
minor bugfixes
matthewpeterkort Nov 9, 2023
3066689
Adds tqdm to logging file and fixes drs_object tqdm structure
matthewpeterkort Nov 13, 2023
76ae014
Add pyinstaller with startup message
lbeckman314 Nov 13, 2023
764fe46
Add tmate step to macOS build
lbeckman314 Nov 13, 2023
52b4d68
Updating startup message
lbeckman314 Nov 13, 2023
54fb973
Make gen3 manager compatible with rest of client
matthewpeterkort Jun 25, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .flake8
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
[flake8]
per-file-ignores =
tests/integration/test_problems.py:E501
max-line-length=120
ignore = W605

exclude =
__pycache__
Expand Down
9 changes: 3 additions & 6 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,10 +54,7 @@ jobs:

- name: Unit tests (No server dependencies) with pytest
run: pytest tests/unit --cov=drs_downloader --cov-report term-missing

# - name: Unit and Integration tests with pytest
# run: pytest tests/unit tests/integration --cov=drs_downloader --cov-report term-missing


build-linux:
needs: test
runs-on: ubuntu-latest
Expand All @@ -74,7 +71,7 @@ jobs:
with:
python-version: "3.10"

- name: Install dependencies
- name: Build executable
shell: bash
run: |
python -m venv venv
Expand Down Expand Up @@ -104,7 +101,7 @@ jobs:
with:
python-version: "3.10"

- name: Install dependencies
- name: Build executable
shell: bash
run: |
python -m venv venv
Expand Down
Binary file removed drs_downloader/.DS_Store
Binary file not shown.
15 changes: 15 additions & 0 deletions drs_downloader/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,18 @@
DEFAULT_MAX_SIMULTANEOUS_DOWNLOADERS = 10
DEFAULT_MAX_SIMULTANEOUS_PART_HANDLERS = 3
DEFAULT_PART_SIZE = 10 * MB


def check_for_AnVIL_URIS(uris_list: list[str]) -> bool:
for uri in uris_list:
if is_AnVIL_URI(uri):
return True


def is_AnVIL_URI(uri: str) -> bool:
if isinstance(uri, str):
# Making the whole uri lowercase shouldn't matter since uuids are lowercase anyway
uri = uri.lower()
if (uri.startswith("drs://drs.anv0:") or uri.startswith("drs://dg.anv0:")):
return True
return False
Loading
Loading