Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(l1): parallelize node requests during snap sync healing phase #1872

Merged
merged 292 commits into from
Feb 4, 2025

Conversation

fmoletta
Copy link
Contributor

@fmoletta fmoletta commented Feb 4, 2025

Motivation
Spawn multiple concurrent fetch processes for state & storage nodes (similar to storage ranges) in order to speed up healing phase.

Description

  • Add MAX_PARALLEL_FETCHES constant to limit and customize how many concurrent fetch processes can be spawned at a time when fetching storage ranges, state nodes and storage nodes
  • Spawn multiple sub-processes at a time to fetch state & storage nodes
  • Use recv_many on the storage healer to avoid piling up storage heal requests

Closes #issue_number

Copy link

github-actions bot commented Feb 4, 2025

| File                                                                  | Lines | Diff |
+-----------------------------------------------------------------------+-------+------+
| /home/runner/work/ethrex/ethrex/crates/networking/p2p/peer_handler.rs | 514   | -1   |
+-----------------------------------------------------------------------+-------+------+
| /home/runner/work/ethrex/ethrex/crates/networking/p2p/sync.rs         | 818   | +49  |
+-----------------------------------------------------------------------+-------+------+

Total lines added: +49
Total lines removed: 1
Total lines changed: 50

@fmoletta fmoletta marked this pull request as ready for review February 4, 2025 18:35
@fmoletta fmoletta requested a review from a team as a code owner February 4, 2025 18:35
trie.state_mut().write_node(node, hash)?;
}
// Send storage & bytecode requests
if !hahsed_addresses.is_empty() {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: hashed_addresses

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch! fixed all instances

@fmoletta fmoletta enabled auto-merge February 4, 2025 19:30
@fmoletta fmoletta added this pull request to the merge queue Feb 4, 2025
Merged via the queue into main with commit 32e0890 Feb 4, 2025
19 checks passed
@fmoletta fmoletta deleted the paralell-heal-batches branch February 4, 2025 19:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants