Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support using GCS for backup uploader #1130

Open
wants to merge 75 commits into
base: main
Choose a base branch
from

Conversation

JeffreyDallas
Copy link
Contributor

@JeffreyDallas JeffreyDallas commented Jan 8, 2025

Description

This pull request changes the following:

  • Add new flags for using backupBucket and google credentials
  • google credentials is loaded to newly created k8 secret
  • Update test workflow

Related Issues

Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
…google-storage

Signed-off-by: Jeffrey Tang <[email protected]>

# Conflicts:
#	version.ts
@JeffreyDallas JeffreyDallas requested review from a team and leninmehedy as code owners January 8, 2025 18:37
@JeffreyDallas JeffreyDallas marked this pull request as draft January 8, 2025 18:37
@JeffreyDallas JeffreyDallas changed the title 01101 d backup uploader feat: support using GCS for backup uploader Jan 8, 2025
Copy link
Contributor

github-actions bot commented Jan 8, 2025

Unit Test Results - Linux

  1 files  ±0   58 suites  ±0   4s ⏱️ -1s
227 tests ±0  227 ✅ ±0  0 💤 ±0  0 ❌ ±0 
232 runs  ±0  232 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit 35fa101. ± Comparison against base commit ed6527b.

♻️ This comment has been updated with latest results.

Copy link
Contributor

github-actions bot commented Jan 8, 2025

Unit Test Results - Windows

  1 files  ±0   58 suites  ±0   12s ⏱️ -1s
227 tests ±0  227 ✅ ±0  0 💤 ±0  0 ❌ ±0 
232 runs  ±0  232 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit 759cd04. ± Comparison against base commit ed6527b.

♻️ This comment has been updated with latest results.

Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
@JeffreyDallas JeffreyDallas marked this pull request as ready for review January 9, 2025 00:04
JeffreyDallas and others added 4 commits January 8, 2025 18:04
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Copy link
Contributor

github-actions bot commented Jan 9, 2025

E2E Test Report

 16 files  121 suites   1h 21m 23s ⏱️
258 tests 258 ✅ 0 💤 0 ❌
261 runs  261 ✅ 0 💤 0 ❌

Results for commit 48d1a0d.

Copy link
Contributor

github-actions bot commented Jan 9, 2025

E2E Test Report

 16 files  ±0  121 suites  ±0   1h 25m 35s ⏱️ + 4m 10s
258 tests ±0  258 ✅ ±0  0 💤 ±0  0 ❌ ±0 
261 runs  ±0  261 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit 759cd04. ± Comparison against base commit ed6527b.

♻️ This comment has been updated with latest results.

Copy link

codacy-production bot commented Jan 9, 2025

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation Diff coverage
-0.06% (target: -1.00%) 63.64%
Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (ed6527b) 20607 17398 84.43%
Head commit (759cd04) 20688 (+81) 17455 (+57) 84.37% (-0.06%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#1130) 55 35 63.64%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

See your quality gate settings    Change summary preferences

Codacy stopped sending the deprecated coverage status on June 5th, 2024. Learn more

Copy link

codecov bot commented Jan 9, 2025

Codecov Report

Attention: Patch coverage is 62.96296% with 20 lines in your changes missing coverage. Please review.

Project coverage is 84.34%. Comparing base (1cd0e28) to head (48d1a0d).
Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
src/commands/network.ts 31.03% 20 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1130      +/-   ##
==========================================
- Coverage   84.66%   84.34%   -0.32%     
==========================================
  Files          75       75              
  Lines       20302    20507     +205     
  Branches     1929     1458     -471     
==========================================
+ Hits        17188    17297     +109     
- Misses       2957     3156     +199     
+ Partials      157       54     -103     
Files with missing lines Coverage Δ
src/commands/flags.ts 74.94% <100.00%> (+0.33%) ⬆️
src/core/constants.ts 97.23% <100.00%> (+0.02%) ⬆️
src/commands/network.ts 66.83% <31.03%> (-1.09%) ⬇️

... and 24 files with indirect coverage changes

Impacted file tree graph

mishomihov00
mishomihov00 previously approved these changes Jan 9, 2025
Copy link
Contributor

@mishomihov00 mishomihov00 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review applies to .github/workflows/flow-gcs-test.yaml

Signed-off-by: Jeffrey Tang <[email protected]>
…backup-uploader

Signed-off-by: Jeffrey Tang <[email protected]>

# Conflicts:
#	src/commands/flags.ts
#	src/commands/network.ts
#	version.ts
jeromy-cannon
jeromy-cannon previously approved these changes Jan 9, 2025
exit 1
fi

echo "${GCP_SERVICE_ACCOUNT_TOKEN}" > gcp_service_account.json
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this also echo to console?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, this is to save to file so it can be loaded by solo command

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add S3 and GCS support into the Backup Uploader in solo-charts and update Solo to provide the secret
5 participants