Skip to content

Commit

Permalink
feat(l1): improve storage download queue management during snap sync (#…
Browse files Browse the repository at this point in the history
…1828)

**Motivation**
Accounts are downloaded at a much faster pace compared to storages,
causing bottlenecks in storage download. A previous PR sped up storage
download by parallelizing requests but we still had many requests piling
up in set ups with multiple peers. This PR aims to alleviate this issue
by reading multiple incoming requests at once and avoid piling requests
in the storage fetcher's receiver
<!-- Why does this pull request exist? What are its goals? -->

**Description**
* Read multiple messages at once from the storage fetcher's receiver
channel during snap sync
<!-- A clear and concise general description of the changes this PR
introduces -->

<!-- Link to issues: Resolves #111, Resolves #222 -->

Closes #issue_number
  • Loading branch information
fmoletta authored Jan 31, 2025
1 parent 17a47c5 commit 6f4f219
Showing 1 changed file with 12 additions and 5 deletions.
17 changes: 12 additions & 5 deletions crates/networking/p2p/sync.rs
Original file line number Diff line number Diff line change
Expand Up @@ -545,12 +545,19 @@ async fn storage_fetcher(
let mut incoming = true;
while incoming {
// Fetch incoming requests
match receiver.recv().await {
Some(account_hashes_and_roots) if !account_hashes_and_roots.is_empty() => {
pending_storage.extend(account_hashes_and_roots);
let mut msg_buffer = vec![];
if receiver.recv_many(&mut msg_buffer, 25).await != 0 {
for account_hashes_and_roots in msg_buffer {
if !account_hashes_and_roots.is_empty() {
pending_storage.extend(account_hashes_and_roots);
} else {
// Empty message signaling no more bytecodes to sync
incoming = false
}
}
// Disconnect / Empty message signaling no more bytecodes to sync
_ => incoming = false,
} else {
// Disconnect
incoming = false
}
// If we have enough pending bytecodes to fill a batch
// or if we have no more incoming batches, spawn a fetch process
Expand Down

0 comments on commit 6f4f219

Please sign in to comment.