-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
peer, main, netsync, blockchain: parallel block downloads #2226
base: master
Are you sure you want to change the base?
peer, main, netsync, blockchain: parallel block downloads #2226
Commits on Aug 20, 2024
-
peer: make peer meet query.Peer interface
query.Peer is used for downloading blocks out of order during headers first download. Methods SubscribeRecvMsg() and OnDisconnect() are added to abide by the interface.
Configuration menu - View commit details
-
Copy full SHA for 782dd2f - Browse repository at this point
Copy the full SHA 782dd2fView commit details -
main: add ConnectedPeers() to server
ConnectedPeers returns all the currently connected peers. This is used to provide the query.WorkManager with all the currently connected peers from the netsync package.
Configuration menu - View commit details
-
Copy full SHA for 510083d - Browse repository at this point
Copy the full SHA 510083dView commit details
Commits on Sep 6, 2024
-
netsync: add checkpointedBlocksQuery
checkpointedBlocksQuery is a helper to create []*query.Request which can be passed off to query.Workmanager to query for wire.Messages to multiple peers. This is useful for downloading blocks out of order from multiple peers during ibd.
Configuration menu - View commit details
-
Copy full SHA for eebcd60 - Browse repository at this point
Copy the full SHA eebcd60View commit details -
netsync: refactor handleBlockMsg
handleBlockMsg used to check that the block header is both valid and then process the blocks as they come in. It's now refactored so that it also handles blocks that are not in order. For out of order block downloads handleBlockMsg would mark the block as an orphan but it's now refactored to handle those cases. Whenever a block that's not the next from the chain tip is received, it's now temporarily stored in memory until the next block from the chain tip is received. And then all the blocks that are in sequence are processed.
Configuration menu - View commit details
-
Copy full SHA for e3e450f - Browse repository at this point
Copy the full SHA e3e450fView commit details -
peerSubscription is added to Manager which will allow it subscribers to receive peers through the channel whenever the Manager is aware of a new peer that it's been connected to. This is useful to alert query.Workmanager that a new peer that's been connected to is eligible to download blocks from.
Configuration menu - View commit details
-
Copy full SHA for 28d9f2b - Browse repository at this point
Copy the full SHA 28d9f2bView commit details -
netsync, main: add ConnectedPeers to Manager
ConnectedPeers returns all the currently connected peers and any new peer that's additionally connected through the returned channel. This method is required for query.Workmanager as it needs ot receive peers that it can request blocks from.
Configuration menu - View commit details
-
Copy full SHA for ee0f49d - Browse repository at this point
Copy the full SHA ee0f49dView commit details -
netsync: add fetchManager and request blocks from it
The blocks that were requested from headers are now sent over to query.Workmanager where it will rank peers based on their speed and request blocks from them accordingly. This allows for quicker block downloads as: 1: Workmanager will prioritize faster peers. 2: Workmanager is able to ask from multiple peers.
Configuration menu - View commit details
-
Copy full SHA for 159cd2d - Browse repository at this point
Copy the full SHA 159cd2dView commit details -
blockchain: store block concurrently while verifying it
Storing block happens before the block validation is done and this can be a bottleneck on computers with slow disks. Allowing for concurrent block storage saves time as the disk operation can be done in parallel with the cpu operations of verifying the block.
Configuration menu - View commit details
-
Copy full SHA for ed15b8a - Browse repository at this point
Copy the full SHA ed15b8aView commit details -
netsync: add logger for blocks downloaded from different peers during
headers-first block download
Configuration menu - View commit details
-
Copy full SHA for 7c1b80f - Browse repository at this point
Copy the full SHA 7c1b80fView commit details -
netsync: don't reset the requestedBlocks in headersFirst
Resetting the requestedBlocks state in headersFirst is problematic since we may be banning peers that are still good.
Configuration menu - View commit details
-
Copy full SHA for d8d8dd7 - Browse repository at this point
Copy the full SHA d8d8dd7View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2cbb562 - Browse repository at this point
Copy the full SHA 2cbb562View commit details