Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize state sync snapshot format #3701

Merged
merged 7 commits into from
Aug 28, 2024
Merged

Optimize state sync snapshot format #3701

merged 7 commits into from
Aug 28, 2024

Conversation

sug0
Copy link
Contributor

@sug0 sug0 commented Aug 26, 2024

Describe your changes

These changes optimize the format of snapshots taken for state syncing purposes. Snapshots are taken over the entire RocksDB database, packaged into a zstd compressed tar archive, and split into 10 MB chunks.

Checklist before merging

  • If this PR has some consensus breaking changes, I added the corresponding breaking:: labels
    • This will require 2 reviewers to approve the changes

@sug0 sug0 added the enhancement New feature or request label Aug 26, 2024
@sug0 sug0 requested a review from tzemanovic August 26, 2024 14:26
@sug0 sug0 force-pushed the bat/snapshot-improvements branch 2 times, most recently from 6ce1cd2 to 353a79b Compare August 26, 2024 18:25
Copy link

codecov bot commented Aug 26, 2024

Codecov Report

Attention: Patch coverage is 73.79310% with 114 lines in your changes missing coverage. Please review.

Project coverage is 61.50%. Comparing base (9274550) to head (a2e067f).
Report is 8 commits behind head on main.

Files Patch % Lines
crates/node/src/shell/snapshots.rs 0.00% 50 Missing ⚠️
crates/node/src/storage/rocksdb.rs 81.50% 49 Missing ⚠️
crates/node/src/shims/abcipp_shim.rs 0.00% 8 Missing ⚠️
crates/node/src/shell/mod.rs 95.45% 5 Missing ⚠️
crates/node/src/shims/abcipp_shim_types.rs 0.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3701      +/-   ##
==========================================
+ Coverage   61.25%   61.50%   +0.25%     
==========================================
  Files         315      315              
  Lines      103196   103880     +684     
==========================================
+ Hits        63209    63896     +687     
+ Misses      39987    39984       -3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@sug0 sug0 force-pushed the bat/snapshot-improvements branch 2 times, most recently from 5caa1fc to 9d83731 Compare August 27, 2024 09:09
@tzemanovic
Copy link
Member

@Mergifyio refresh

Copy link
Contributor

mergify bot commented Aug 27, 2024

refresh

✅ Pull request refreshed

@tzemanovic
Copy link
Member

@Mergifyio refresh

Copy link
Contributor

mergify bot commented Aug 27, 2024

refresh

✅ Pull request refreshed

@sug0 sug0 force-pushed the bat/snapshot-improvements branch from 705d951 to f027fca Compare August 28, 2024 09:47
@sug0 sug0 force-pushed the bat/snapshot-improvements branch from f027fca to a2e067f Compare August 28, 2024 09:49
@sug0
Copy link
Contributor Author

sug0 commented Aug 28, 2024

@Mergifyio refresh

Copy link
Contributor

mergify bot commented Aug 28, 2024

refresh

✅ Pull request refreshed

@sug0 sug0 added the merge Ready to merge - mergifyio bot will add the PR to merge queue when all checks pass label Aug 28, 2024
@mergify mergify bot merged commit 62be903 into main Aug 28, 2024
22 checks passed
@mergify mergify bot deleted the bat/snapshot-improvements branch August 28, 2024 10:27
@sug0 sug0 mentioned this pull request Aug 29, 2024
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request merge Ready to merge - mergifyio bot will add the PR to merge queue when all checks pass
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants