Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update downloader to read allow lists from configs #1484

Merged
merged 1 commit into from
Mar 2, 2024

Conversation

ntascii
Copy link
Contributor

@ntascii ntascii commented Mar 1, 2024

Summary

Updated download to read allow list configs from /etc/deploy/deployagent.conf.
Added a bazel target for downloader
Updated tests

Test plan

UTs

bazel test tests/test_http_download_helper
bazel test tests/test_s3_download_helper

test downloader, empty config

sudo bazel build deployd:deploy-downloader 
sudo bazel-bin/deployd/deploy-downloader -f /etc/deployagent.conf -v 2bhd_k2QT5GjAx0aR0G2mA_aa874a6 -u https://deployrepo.pinadmin.com/deploy-agent-pex/deploy-agent-pex-aa874a6.tar.gz -e deploy-init
2024-03-01 23:38:00,945:INFO: Start to download from url https://deployrepo.pinadmin.com/deploy-agent-pex/deploy-agent-pex-aa874a6.tar.gz to /mnt/builds/deploy-init-2bhd_k2QT5GjAx0aR0G2mA_aa874a6.tar.gz
2024-03-01 23:38:00,946:INFO: Running command: curl -o /mnt/builds/deploy-init-2bhd_k2QT5GjAx0aR0G2mA_aa874a6.tar.gz -fksS https://deployrepo.pinadmin.com/deploy-agent-pex/deploy-agent-pex-aa874a6.tar.gz
2024-03-01 23:38:01,073:INFO: Finish downloading: https://deployrepo.pinadmin.com/deploy-agent-pex/deploy-agent-pex-aa874a6.tar.gz to /mnt/builds/deploy-init-2bhd_k2QT5GjAx0aR0G2mA_aa874a6.tar.gz
2024-03-01 23:38:01,077:DEBUG: Starting new HTTPS connection (1): deployrepo.pinadmin.com:443
2024-03-01 23:38:01,107:DEBUG: https://deployrepo.pinadmin.com:443 "GET /deploy-agent-pex/deploy-agent-pex-aa874a6.tar.gz.sha1 HTTP/1.1" 404 324
2024-03-01 23:38:01,107:WARNING: Skip checksum verification. Invalid response from https://deployrepo.pinadmin.com/deploy-agent-pex/deploy-agent-pex-aa874a6.tar.gz.sha1
2024-03-01 23:38:01,108:INFO: untar files to /mnt/builds/2bhd_k2QT5GjAx0aR0G2mA_aa874a6
2024-03-01 23:38:01,317:INFO: Successfully extracted /mnt/builds/deploy-init-2bhd_k2QT5GjAx0aR0G2mA_aa874a6.tar.gz to /mnt/builds/2bhd_k2QT5GjAx0aR0G2mA_aa874a6
2024-03-01 23:38:01,317:INFO: Download succeeded.

Test downloader, mocking config

2024-03-01 23:39:16,374:INFO: Start to download from url https://deployrepo.pinadmin.com/deploy-agent-pex/deploy-agent-pex-aa874a6.tar.gz to /mnt/builds/deploy-init-2bhd_k2QT5GjAx0aR0G2mA_aa874a6.tar.gz
2024-03-01 23:39:16,374:ERROR: deployrepo.pinadmin.com is not in the allow list: ['test.com'].
2024-03-01 23:39:16,374:ERROR: Invalid url: https://deployrepo.pinadmin.com/deploy-agent-pex/deploy-agent-pex-aa874a6.tar.gz. Skip downloading.
2024-03-01 23:39:16,374:ERROR: Download failed.

@github-actions github-actions bot added the deploy-agent Includes changes to deploy-agent label Mar 1, 2024
@ntascii ntascii marked this pull request as ready for review March 1, 2024 23:41
@ntascii ntascii requested a review from a team as a code owner March 1, 2024 23:41
@ntascii ntascii merged commit 0d13631 into master Mar 2, 2024
7 checks passed
@ntascii ntascii deleted the anhn/download-alow-list branch March 2, 2024 08:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
deploy-agent Includes changes to deploy-agent
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants