Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: use gcs for stream file storage #994

Merged
merged 58 commits into from
Jan 7, 2025
Merged

Conversation

JeffreyDallas
Copy link
Contributor

@JeffreyDallas JeffreyDallas commented Dec 17, 2024

Description

This pull request changes the following:

  • New flags for cloud storage type, keys and endpoint
  • Generate new secrets with solo
  • Refactor create-topic example to also check submitted message from mirror node
  • move create_test_account to individual helper file to shared between solo relay test and gcs test
  • Add test workflow to verify using google storage for stream files upload and download

Related Issues

Test result

https://github.com/hashgraph/solo/actions/runs/12473809302/job/34814887772

Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
@JeffreyDallas JeffreyDallas requested review from a team as code owners December 17, 2024 03:02
@JeffreyDallas JeffreyDallas self-assigned this Dec 17, 2024
@JeffreyDallas JeffreyDallas marked this pull request as draft December 17, 2024 03:02
Copy link
Contributor

github-actions bot commented Dec 17, 2024

Unit Test Results - Linux

  1 files  ±0   58 suites  ±0   3s ⏱️ -1s
227 tests ±0  227 ✅ ±0  0 💤 ±0  0 ❌ ±0 
232 runs  ±0  232 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit f29bdf6. ± Comparison against base commit 5b9d937.

♻️ This comment has been updated with latest results.

Copy link
Contributor

github-actions bot commented Dec 17, 2024

Unit Test Results - Windows

  1 files  ±0   58 suites  ±0   15s ⏱️ -1s
227 tests ±0  227 ✅ ±0  0 💤 ±0  0 ❌ ±0 
232 runs  ±0  232 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit f29bdf6. ± Comparison against base commit 5b9d937.

♻️ This comment has been updated with latest results.

Copy link
Contributor

E2E Test Report

 16 files  ±0  121 suites  ±0   1h 18m 41s ⏱️ - 2m 23s
256 tests ±0  256 ✅ ±0  0 💤 ±0  0 ❌ ±0 
259 runs  ±0  259 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit ca46661. ± Comparison against base commit a5fab1e.

Copy link
Contributor

github-actions bot commented Dec 17, 2024

E2E Test Report

 16 files  121 suites   1h 23m 19s ⏱️
258 tests 258 ✅ 0 💤 0 ❌
261 runs  261 ✅ 0 💤 0 ❌

Results for commit f29bdf6.

♻️ This comment has been updated with latest results.

Copy link

codacy-production bot commented Dec 17, 2024

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation Diff coverage
Report missing for 5b9d9371 60.78%
Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (5b9d937) Report Missing Report Missing Report Missing
Head commit (f29bdf6) 20293 17353 85.51%

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#994) 255 155 60.78%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

See your quality gate settings    Change summary preferences

Codacy stopped sending the deprecated coverage status on June 5th, 2024. Learn more

Footnotes

  1. Codacy didn't receive coverage data for the commit, or there was an error processing the received data. Check your integration for errors and validate that your coverage setup is correct.

Copy link

codecov bot commented Dec 17, 2024

Codecov Report

Attention: Patch coverage is 60.00000% with 102 lines in your changes missing coverage. Please review.

Project coverage is 84.67%. Comparing base (362df60) to head (f29bdf6).
Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
src/commands/network.ts 40.44% 81 Missing ⚠️
src/commands/mirror_node.ts 44.44% 20 Missing ⚠️
src/core/config_manager.ts 88.88% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #994      +/-   ##
==========================================
- Coverage   85.03%   84.67%   -0.36%     
==========================================
  Files          75       75              
  Lines       20043    20293     +250     
  Branches     1333     1286      -47     
==========================================
+ Hits        17044    17184     +140     
- Misses       2946     3092     +146     
+ Partials       53       17      -36     
Files with missing lines Coverage Δ
src/commands/flags.ts 74.61% <100.00%> (+0.89%) ⬆️
src/core/constants.ts 97.20% <100.00%> (+0.15%) ⬆️
src/core/k8.ts 87.95% <100.00%> (-0.50%) ⬇️
version.ts 100.00% <100.00%> (ø)
src/core/config_manager.ts 94.27% <88.88%> (-0.27%) ⬇️
src/commands/mirror_node.ts 73.90% <44.44%> (-1.70%) ⬇️
src/commands/network.ts 67.92% <40.44%> (-4.56%) ⬇️

... and 18 files with indirect coverage changes

Impacted file tree graph

Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
@JeffreyDallas JeffreyDallas marked this pull request as ready for review December 19, 2024 16:30
Signed-off-by: Jeffrey Tang <[email protected]>
@JeffreyDallas JeffreyDallas marked this pull request as draft December 19, 2024 17:05
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
mishomihov00
mishomihov00 previously approved these changes Jan 6, 2025
Copy link
Contributor

@mishomihov00 mishomihov00 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review applies to:
.github/workflows/flow-gcs-test.yaml

src/commands/flags.ts Outdated Show resolved Hide resolved
…google-storage

Signed-off-by: Jeffrey Tang <[email protected]>

# Conflicts:
#	src/commands/network.ts
JeffreyDallas and others added 7 commits January 6, 2025 09:11
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Co-authored-by: Jeromy Cannon <[email protected]>
Signed-off-by: JeffreyDallas <[email protected]>
Co-authored-by: Jeromy Cannon <[email protected]>
Signed-off-by: JeffreyDallas <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
mishomihov00
mishomihov00 previously approved these changes Jan 7, 2025
Copy link
Contributor

@mishomihov00 mishomihov00 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review applies to:
.github/workflows/flow-gcs-test.yaml

Signed-off-by: Jeromy Cannon <[email protected]>
jeromy-cannon
jeromy-cannon previously approved these changes Jan 7, 2025
Signed-off-by: Jeffrey Tang <[email protected]>
@jeromy-cannon jeromy-cannon merged commit 04d5806 into main Jan 7, 2025
41 of 42 checks passed
@jeromy-cannon jeromy-cannon deleted the 00711-D-google-storage branch January 7, 2025 18:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Charts[Network]: expose variables to values.yaml for enabling and configuring mirror s3 and gcs backups
3 participants