-
Notifications
You must be signed in to change notification settings - Fork 37
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(l1): snap sync overhaul #1763
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…to save-peer-capabilities
…to add-holesky-presets
…to add-sepolia-presets
ilitteri
reviewed
Jan 22, 2025
ilitteri
reviewed
Jan 22, 2025
mpaulucci
approved these changes
Jan 23, 2025
github-merge-queue bot
pushed a commit
that referenced
this pull request
Feb 3, 2025
**Motivation** Reverts the "slow but safe" approach to healing taken by #1763 and instead adds the intermediate nodes received from peers to the trie. <!-- Why does this pull request exist? What are its goals? --> **Description** * Write nodes fetched during healing to the state instead of writing only leaf values * Track paths left in queue when a healing cycle ends due to pivot staleness * Condense snap checkpoint clearing to one single method * (Misc) Move some noisy p2p info tracing to debug * (Fix) When executing full blocks after the pivot block during snap sync execute the block first before setting it as canonical <!-- A clear and concise general description of the changes this PR introduces --> <!-- Link to issues: Resolves #111, Resolves #222 --> **Comments** A not so pretty solution was implemented on the storage_healer (the startup flag) to avoid blocking storage healing at the start. This can and will be improved but doing so in this PR is not worth it when follow up PRs will already touch up on this logic when parallelizing storage node fetching (similar to storage ranges) **Testing** This proved to work fine in `Mekong` testnet when introducing forced sleeps and delays to force healing. But it is still too slow to work on Holesky testnet, this should improve once we add more parallelization to state sync & healing Closes #issue_number
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
This PR introduces the following upgrades for snap-sync:
And also simplifies it by removing the following logic:
Description
Stores the last downloaded block's hash in the DB during snap sync to serve as a checkpoint if the sync is aborted halfway (common case when syncing from genesis). This checkpoint is cleared upon succesful snap sync.
No longer fetches receipts or block bodies past the pivot block during snap sync
Add method
sync_status
which returns an enum with the current sync status (either Inactive, Active or Pending) and uses it in the ForkChoiceUpdate & NewPayload engine rpc endpoints so that we don't apply their logic during an active or pending sync.Fetcher process now identify stale pivots and remain passive until they receive the end signal
Fetcher processes now return their current queue upon return so that it can be persisted into the next cycle
Stores the latest state root during state sync and healing as a checkpoint
Stores the last fetched key during state sync as a checkpoint
Healing no longer stores the nodes received via p2p, it instead inserts the leaf values and rebuilds it to avoid trie corruption between restarts.
The current progress percentage and estimated time to finish is periodically reported during state sync
Disables the following Paris & Cancun engine hive tests that previously yielded false positives due to new payloads being accepted on top of a syncing chain:
And also disables the following tests that fail with the flag Syncing=true for the same reason :
Misc:
Closes None
Closes #issue_number