Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(dot/sync): include block origin and skip extra validation on initialSync #3392

Merged
merged 144 commits into from
Nov 29, 2023
Merged
Changes from 1 commit
Commits
Show all changes
144 commits
Select commit Hold shift + click to select a range
f3eb2eb
wip: implementing a poc around warp sync
EclesioMeloJunior Mar 30, 2023
d7c07ef
chore: include informative logs to debug sync
EclesioMeloJunior Apr 6, 2023
428ff13
chore: enable state trace logs and improve error for unfinalized ance…
EclesioMeloJunior Apr 11, 2023
25b30d5
wip: handle workers results
EclesioMeloJunior Apr 13, 2023
d05fdec
wip: handling workers results
EclesioMeloJunior Apr 13, 2023
6e272a6
chore: start ignoring peers + use `idleTicker` to check the pool
EclesioMeloJunior Apr 14, 2023
39dd7c3
fix: use buffered channel for taskQueue + wip: implementing tip sync
EclesioMeloJunior Apr 15, 2023
fc081e1
chore: enable state/grandpa/digest trace to investigate forced changes
EclesioMeloJunior Apr 17, 2023
7bc8038
chore: making changes to handling forced changes
EclesioMeloJunior Apr 18, 2023
0a27158
chore: reset scheduled changes once a forced change applies
EclesioMeloJunior Apr 18, 2023
bf6bb61
chore: rollback basepath
EclesioMeloJunior Apr 19, 2023
59cbaae
chore: wip tip sync
EclesioMeloJunior Apr 19, 2023
f9ca5a4
wip: tip sync impl
EclesioMeloJunior Apr 24, 2023
f9d7e43
chore: complete tip sync adjusts
EclesioMeloJunior Apr 26, 2023
3ba7e4c
chore: rollback modification on .yml files
EclesioMeloJunior Apr 26, 2023
16e8ee6
chore: rollback modifications to digest pkg
EclesioMeloJunior Apr 26, 2023
3eba164
chore: rollback modification to network pkg
EclesioMeloJunior Apr 26, 2023
c30948a
chore: rollback modifications to auth set changes
EclesioMeloJunior Apr 26, 2023
57f433c
chore: remove `internal/sync` pkg
EclesioMeloJunior Apr 26, 2023
a76c0e9
chore: remove unneeded comments
EclesioMeloJunior Apr 26, 2023
0bd8505
chore: remove unneeded fields from sync worker
EclesioMeloJunior Apr 27, 2023
5102e62
chore: remove dead code + add errors wrappers
EclesioMeloJunior Apr 27, 2023
c13b597
chore: address some points
EclesioMeloJunior Apr 27, 2023
f0c667f
chore: fix missing blocks in ascedingBlockRequests
EclesioMeloJunior May 2, 2023
f3cb072
Merge branch 'development' into eclesio/fix-bootstrap-sync
EclesioMeloJunior May 2, 2023
2e640ce
chore: delete chainprocessor.go from merge with development
EclesioMeloJunior May 2, 2023
683e331
chore: impl wait/release while validating a worker result
EclesioMeloJunior May 3, 2023
b3d4c8e
wip: tip sync concurrency for bounded requests
EclesioMeloJunior May 5, 2023
c446fc0
wip: bounded requests without blocking the task queue listener
EclesioMeloJunior May 6, 2023
938d001
Merge branch 'development' into eclesio/fix-bootstrap-sync
EclesioMeloJunior May 6, 2023
c8f47fa
chore: use versioned config data + enable traces
EclesioMeloJunior May 6, 2023
ce40f14
chore: remove state trace
EclesioMeloJunior May 6, 2023
e33c444
chore: revert changes from `westend-local-spec.json`
EclesioMeloJunior May 6, 2023
f00a242
chore: enable trace to `sync` and `digest`
EclesioMeloJunior May 6, 2023
6e3d2d0
chore: enable pprof to staging env
EclesioMeloJunior May 8, 2023
023df2c
chore: use `l.Lock()` instead of `l.RLock()`
EclesioMeloJunior May 8, 2023
0f661de
chore: change the worker_pool
EclesioMeloJunior May 8, 2023
924d172
chore: small fixes to the `executeRequest` method
EclesioMeloJunior May 9, 2023
6b481c1
chore: remove unneeded comment line + improve func name
EclesioMeloJunior May 9, 2023
539e092
Merge branch 'development' into eclesio/fix-bootstrap-sync
EclesioMeloJunior May 18, 2023
9ae3fcb
Merge branch 'development' into eclesio/fix-bootstrap-sync
EclesioMeloJunior May 19, 2023
35c9eea
chore: re-written `Test_chainSync_setBlockAnnounce` test
EclesioMeloJunior May 22, 2023
443045c
chore: include a bootstrap sync test with 2 workers
EclesioMeloJunior May 23, 2023
2432324
chore: testing successful sync with workers failing
EclesioMeloJunior May 24, 2023
5c7f983
chore: adjust mocks
EclesioMeloJunior May 24, 2023
17b03b7
chore: avoid ignoring peers whenever we failed to read response bytes
EclesioMeloJunior May 24, 2023
c9d16ce
Merge branch 'development' into eclesio/fix-bootstrap-sync
EclesioMeloJunior May 25, 2023
0f69520
chore: add export comment to `AllConnectedPeers`
EclesioMeloJunior May 25, 2023
ae202b9
Merge branch 'eclesio/fix-bootstrap-sync' of github.com:ChainSafe/gos…
EclesioMeloJunior May 25, 2023
e780937
chore: improve readability + ignore peers management
EclesioMeloJunior May 29, 2023
4cd4fe5
Merge branch 'development' into eclesio/fix-bootstrap-sync
EclesioMeloJunior May 29, 2023
ae8b367
chore: make the peer discovery everytime
EclesioMeloJunior May 29, 2023
e1b4bd4
chore: use a resource manager to autoscale outbound requests
EclesioMeloJunior May 30, 2023
a26b845
chore: remove `maxRequestsAllowed` restriction
EclesioMeloJunior May 30, 2023
e978240
Merge branch 'development' into eclesio/fix-bootstrap-sync
EclesioMeloJunior May 30, 2023
90606ab
chore: remove unneeded deltas on westend chain
EclesioMeloJunior May 30, 2023
795e1b7
Merge branch 'eclesio/fix-bootstrap-sync' of github.com:ChainSafe/gos…
EclesioMeloJunior May 30, 2023
bc88064
chore: addressing comments
EclesioMeloJunior May 30, 2023
9ae5d5a
Merge branch 'development' into eclesio/fix-bootstrap-sync
EclesioMeloJunior May 30, 2023
3103fdb
Merge branch 'development' into eclesio/fix-bootstrap-sync
EclesioMeloJunior May 30, 2023
43ba321
Merge branch 'eclesio/fix-bootstrap-sync' of github.com:ChainSafe/gos…
EclesioMeloJunior May 30, 2023
0e33188
chore: reduce number of max peers
EclesioMeloJunior May 30, 2023
6982f2a
chore: make target number clear on `requesting_4_chunks_of_128_plus_3…
EclesioMeloJunior May 31, 2023
22f62d7
chore: remove unneeded `blockAnnounceCh`
EclesioMeloJunior May 31, 2023
35197c5
chore: reduce the amount of blocks synced at same time
EclesioMeloJunior Jun 1, 2023
576bb72
Merge branch 'development' into eclesio/fix-bootstrap-sync
EclesioMeloJunior Jun 2, 2023
1b991b3
chore: merge development
EclesioMeloJunior Jun 5, 2023
c73789a
Merge branch 'development' into eclesio/fix-sync-speed
EclesioMeloJunior Jun 5, 2023
d9b5d19
chore: remove unneeded sync files
EclesioMeloJunior Jun 5, 2023
5bc45f9
EclesioMeloJunior Jun 6, 2023
5fb3070
chore: use highest finalized header instead of best block number
EclesioMeloJunior Jun 6, 2023
6ee8b4a
chore: using finalised header to bootstrap sync
EclesioMeloJunior Jun 6, 2023
c7e145b
chore: use a checkpoint header every time we failed to import blocks
EclesioMeloJunior Jun 6, 2023
a6086d6
chore: fix `all goroutines are asleep - deadlock` problem
EclesioMeloJunior Jun 7, 2023
19606f1
Merge branch 'development' into eclesio/fix-bootstrap-sync
EclesioMeloJunior Jun 7, 2023
22699e8
chore: fix punishment
EclesioMeloJunior Jun 7, 2023
e0ea946
Merge branch 'development' into eclesio/fix-bootstrap-sync
EclesioMeloJunior Jun 16, 2023
845e42e
chore: fix `executeBootstrapSync` tests
EclesioMeloJunior Jun 16, 2023
61c1a51
chore: revert libp2p resource manager additions
EclesioMeloJunior Jun 16, 2023
f415b90
chore: rename `AllConnectedPeers` to `AllConnectedPeersID`
EclesioMeloJunior Jun 16, 2023
fe132b4
chore: move `ErrReceivedEmptyMessage` to errors file
EclesioMeloJunior Jun 16, 2023
1df21a3
chore: rename `<-done` to `<-stop`
EclesioMeloJunior Jun 16, 2023
eeb5118
chore: create `TestSyncWorkerPool_useConnectedPeers` test
EclesioMeloJunior Jun 20, 2023
10dd486
chore: add methods comments
EclesioMeloJunior Jun 20, 2023
9d2b32d
chore: include `TestSyncWorkerPool_newPeer` test
EclesioMeloJunior Jun 20, 2023
e59228a
chore: rename `l` to `mtx`
EclesioMeloJunior Jun 20, 2023
3e7f420
chore: remove unneeded variable
EclesioMeloJunior Jun 20, 2023
1959231
chore: add `TestSyncWorkerPool__listenForRequests_submitRequest` test
EclesioMeloJunior Jun 20, 2023
4ea26dc
chore: add `TestSyncWorkerPool_listenForRequests_busyWorkers` test
EclesioMeloJunior Jun 20, 2023
a2d45d8
Merge branch 'development' into eclesio/fix-bootstrap-sync
EclesioMeloJunior Jun 20, 2023
7fc622b
Merge branch 'development' into eclesio/fix-bootstrap-sync
EclesioMeloJunior Jun 20, 2023
16de56f
chore: move request message creation to `network` pkg
EclesioMeloJunior Jun 20, 2023
51a78cf
Merge branch 'eclesio/fix-bootstrap-sync' of github.com:ChainSafe/gos…
EclesioMeloJunior Jun 20, 2023
3c2b14f
chore: fix tests to use `network.MaxBlockResponseSize`
EclesioMeloJunior Jun 20, 2023
beb7f8f
chore: fix lint warns
EclesioMeloJunior Jun 20, 2023
6d7d564
chore: adding license to `worker_pool_test.go`
EclesioMeloJunior Jun 20, 2023
4096823
chore: remove unneeded else branch
EclesioMeloJunior Jun 21, 2023
060dbc6
chore: fix deepsource warns
EclesioMeloJunior Jun 21, 2023
dc82d93
chore: wip `announced_block_while_in_tip_mode` test
EclesioMeloJunior Jun 21, 2023
057bacb
Merge branch 'development' into eclesio/fix-bootstrap-sync
EclesioMeloJunior Jun 21, 2023
4a51be3
chore: add `announced_block_while_in_tip_mode` test
EclesioMeloJunior Jun 21, 2023
3d37f61
chore: add `TestChainSync_validateResponse` test
EclesioMeloJunior Jun 21, 2023
b09cdaa
chore: fix `validateBlockResponse` checks
EclesioMeloJunior Jun 21, 2023
1a56e6d
chore: solve all tests failures
EclesioMeloJunior Jun 22, 2023
0f43ae4
Merge branch 'eclesio/fix-bootstrap-sync' of github.com:ChainSafe/gos…
EclesioMeloJunior Jun 22, 2023
2836777
chore: move worker status update from `defer`
EclesioMeloJunior Jun 22, 2023
55ff3f8
chore: solve `lll` lint wanr
EclesioMeloJunior Jun 22, 2023
9ef76cf
chore: improve code coverage by testing failure scenarios
EclesioMeloJunior Jun 22, 2023
7037924
chore: request missing blocks
EclesioMeloJunior Jun 27, 2023
7bbebfd
chore: punish only if is a best block, ignore key not found
EclesioMeloJunior Jul 4, 2023
2a97b9b
Merge branch 'development' into eclesio/fix-bootstrap-sync
EclesioMeloJunior Jul 5, 2023
83a553d
Merge branch 'development' into eclesio/fix-bootstrap-sync
EclesioMeloJunior Jul 5, 2023
990b9c9
chore: add license
EclesioMeloJunior Jul 5, 2023
3c96c08
chore: remove unneeded nil check and fix `lll` lint warn
EclesioMeloJunior Jul 5, 2023
4047178
Merge branch 'eclesio/fix-bootstrap-sync' of github.com:ChainSafe/gos…
EclesioMeloJunior Jul 5, 2023
8d712ca
chore: fix `dot/sync` integration tests
EclesioMeloJunior Jul 5, 2023
22bb4cb
Merge branch 'development' into eclesio/fix-bootstrap-sync
EclesioMeloJunior Jul 5, 2023
4a1905e
chore: fix `TestAscendingBlockRequest` test
EclesioMeloJunior Jul 5, 2023
fbaca64
Merge branch 'eclesio/fix-bootstrap-sync' of github.com:ChainSafe/gos…
EclesioMeloJunior Jul 5, 2023
9e2acb3
chore: fix `TestNewNode` test
EclesioMeloJunior Jul 6, 2023
1610978
chore: fix `TestStartStopNode` test
EclesioMeloJunior Jul 6, 2023
9f540b9
Merge branch 'development' into eclesio/fix-bootstrap-sync
EclesioMeloJunior Jul 6, 2023
bb6e4d7
chore: include `TestChainSync_BootstrapSync_SucessfulSync_ReceivedPar…
EclesioMeloJunior Jul 6, 2023
4741a28
Merge branch 'eclesio/fix-bootstrap-sync' of github.com:ChainSafe/gos…
EclesioMeloJunior Jul 6, 2023
e79f2b4
feat: include block origin in the sync pkg
EclesioMeloJunior Jul 6, 2023
d92e6ce
chore: include get target, remove unneeded print
EclesioMeloJunior Jul 10, 2023
3d21339
chore: wip
EclesioMeloJunior Jul 12, 2023
ef59819
chore: remove unneeded changes
EclesioMeloJunior Jul 19, 2023
fe27c9d
chore: remove unneeded changes
EclesioMeloJunior Jul 19, 2023
c2bdb93
Merge branch 'development' into eclesio/block-origin
EclesioMeloJunior Aug 16, 2023
cb705fe
chore: address comments
EclesioMeloJunior Aug 16, 2023
de80302
Merge branch 'development' into eclesio/block-origin
EclesioMeloJunior Aug 16, 2023
286819e
Merge branch 'development' into eclesio/block-origin
EclesioMeloJunior Aug 17, 2023
ec7da00
Merge branch 'development' into eclesio/block-origin
EclesioMeloJunior Aug 30, 2023
b67b014
Merge branch 'development' into eclesio/block-origin
EclesioMeloJunior Sep 11, 2023
9efe2b1
chore: fix lint by adding `nolint:unparam`
EclesioMeloJunior Sep 11, 2023
c5749fa
Merge branch 'development' into eclesio/block-origin
EclesioMeloJunior Nov 1, 2023
b144850
chore: address comments
EclesioMeloJunior Nov 1, 2023
cee24eb
Merge branch 'development' into eclesio/block-origin
EclesioMeloJunior Nov 16, 2023
02d6efc
chore: introduce block origin to tests
EclesioMeloJunior Nov 16, 2023
8d812aa
chore: make `waitEnoughPeersAndTarget` a blocking function
EclesioMeloJunior Nov 17, 2023
189e3b3
Merge branch 'development' into eclesio/block-origin
EclesioMeloJunior Nov 20, 2023
7950d87
Merge branch 'development' into eclesio/block-origin
EclesioMeloJunior Nov 22, 2023
cfff017
Merge branch 'development' into eclesio/block-origin
EclesioMeloJunior Nov 29, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
chore: add announced_block_while_in_tip_mode test
EclesioMeloJunior committed Jun 21, 2023
commit 4a51be3f6c1f036d13fde7cfd96c7fe3e1efddc6
22 changes: 9 additions & 13 deletions dot/sync/chain_sync.go
Original file line number Diff line number Diff line change
@@ -83,7 +83,7 @@ type ChainSync interface {
// getHighestBlock returns the highest block or an error
getHighestBlock() (highestBlock uint, err error)

onImportBlock(announcedBlock) error
onBlockAnnounce(announcedBlock) error
}

type announcedBlock struct {
@@ -270,7 +270,7 @@ func (cs *chainSync) setPeerHead(who peer.ID, bestHash common.Hash, bestNumber u
}
}

func (cs *chainSync) onImportBlock(announced announcedBlock) error {
func (cs *chainSync) onBlockAnnounce(announced announcedBlock) error {
if cs.pendingBlocks.hasBlock(announced.header.Hash()) {
return fmt.Errorf("%w: block %s (#%d)",
errAlreadyInDisjointSet, announced.header.Hash(), announced.header.Number)
@@ -282,14 +282,14 @@ func (cs *chainSync) onImportBlock(announced announcedBlock) error {
}

syncState := cs.state.Load().(chainSyncState)
if syncState == tip {
return cs.requestImportedBlock(announced)
if syncState != tip {
return nil
}

return nil
return cs.requestAnnouncedBlock(announced)
}

func (cs *chainSync) requestImportedBlock(announce announcedBlock) error {
func (cs *chainSync) requestAnnouncedBlock(announce announcedBlock) error {
peerWhoAnnounced := announce.who
announcedHash := announce.header.Hash()
announcedNumber := announce.header.Number
@@ -407,8 +407,9 @@ func (cs *chainSync) requestForkBlocks(bestBlockHeader, highestFinalizedHeader,
}

func (cs *chainSync) requestPendingBlocks(highestFinalizedHeader *types.Header) error {
logger.Infof("total of pending blocks: %d", cs.pendingBlocks.size())
if cs.pendingBlocks.size() == 0 {
pendingBlocksTotal := cs.pendingBlocks.size()
logger.Infof("total of pending blocks: %d", pendingBlocksTotal)
if pendingBlocksTotal < 1 {
return nil
}

@@ -654,11 +655,6 @@ func (cs *chainSync) handleWorkersResults(
lastBlockInResponse.Header.Number, lastBlockInResponse.Hash)
}

cs.network.ReportPeer(peerset.ReputationChange{
Value: peerset.GossipSuccessValue,
Reason: peerset.GossipSuccessReason,
}, taskResult.who)

for _, blockInResponse := range response.BlockData {
blockExactIndex := blockInResponse.Header.Number - startAtBlock
syncingChain[blockExactIndex] = blockInResponse
107 changes: 96 additions & 11 deletions dot/sync/chain_sync_test.go
Original file line number Diff line number Diff line change
@@ -84,10 +84,14 @@ func Test_chainSync_onImportBlock(t *testing.T) {
const somePeer = peer.ID("abc")

errTest := errors.New("test error")
block2AnnounceHeader := types.NewHeader(common.Hash{}, common.Hash{},
emptyTrieState := storage.NewTrieState(nil)
block1AnnounceHeader := types.NewHeader(common.Hash{}, emptyTrieState.MustRoot(),
common.Hash{}, 1, scale.VaryingDataTypeSlice{})
block2AnnounceHeader := types.NewHeader(block1AnnounceHeader.Hash(), emptyTrieState.MustRoot(),
common.Hash{}, 2, scale.VaryingDataTypeSlice{})

testCases := map[string]struct {
listenForRequests bool
chainSyncBuilder func(ctrl *gomock.Controller) *chainSync
peerID peer.ID
blockAnnounceHeader *types.Header
@@ -140,7 +144,79 @@ func Test_chainSync_onImportBlock(t *testing.T) {
peerID: somePeer,
blockAnnounceHeader: block2AnnounceHeader,
},
//"announced_block_while_in_tip_mode": {},
"announced_block_while_in_tip_mode": {
chainSyncBuilder: func(ctrl *gomock.Controller) *chainSync {
pendingBlocksMock := NewMockDisjointBlockSet(ctrl)
pendingBlocksMock.EXPECT().hasBlock(block2AnnounceHeader.Hash()).Return(false)
pendingBlocksMock.EXPECT().addHeader(block2AnnounceHeader).Return(nil)
pendingBlocksMock.EXPECT().removeBlock(block2AnnounceHeader.Hash())
pendingBlocksMock.EXPECT().size().Return(int(0))

blockStateMock := NewMockBlockState(ctrl)
blockStateMock.EXPECT().
HasHeader(block2AnnounceHeader.Hash()).
Return(false, nil)

blockStateMock.EXPECT().
BestBlockHeader().
Return(block1AnnounceHeader, nil)

blockStateMock.EXPECT().
GetHighestFinalisedHeader().
Return(block2AnnounceHeader, nil)

expectedRequest := network.NewSingleBlockRequestMessage(block2AnnounceHeader.Hash(),
network.BootstrapRequestData)

fakeBlockBody := types.Body([]types.Extrinsic{})
mockedBlockResponse := &network.BlockResponseMessage{
BlockData: []*types.BlockData{
{
Hash: block2AnnounceHeader.Hash(),
Header: block2AnnounceHeader,
Body: &fakeBlockBody,
},
},
}

networkMock := NewMockNetwork(ctrl)
networkMock.EXPECT().
DoBlockRequest(somePeer, expectedRequest).
Return(mockedBlockResponse, nil)

babeVerifierMock := NewMockBabeVerifier(ctrl)
storageStateMock := NewMockStorageState(ctrl)
importHandlerMock := NewMockBlockImportHandler(ctrl)
telemetryMock := NewMockTelemetry(ctrl)

const announceBlock = true
ensureSuccessfulBlockImportFlow(t, block1AnnounceHeader, mockedBlockResponse.BlockData,
blockStateMock, babeVerifierMock, storageStateMock, importHandlerMock, telemetryMock,
announceBlock)

workerPool := newSyncWorkerPool(networkMock)
// include the peer who announced the block in the pool
workerPool.newPeer(somePeer)

state := atomic.Value{}
state.Store(tip)

return &chainSync{
pendingBlocks: pendingBlocksMock,
state: state,
workerPool: workerPool,
network: networkMock,
blockState: blockStateMock,
babeVerifier: babeVerifierMock,
telemetry: telemetryMock,
storageState: storageStateMock,
blockImportHandler: importHandlerMock,
}
},
listenForRequests: true,
peerID: somePeer,
blockAnnounceHeader: block2AnnounceHeader,
},
}

for name, tt := range testCases {
@@ -150,7 +226,13 @@ func Test_chainSync_onImportBlock(t *testing.T) {
ctrl := gomock.NewController(t)

chainSync := tt.chainSyncBuilder(ctrl)
err := chainSync.onImportBlock(announcedBlock{
if tt.listenForRequests {
stopCh := make(chan struct{})
defer close(stopCh)
go chainSync.workerPool.listenForRequests(stopCh)
}

err := chainSync.onBlockAnnounce(announcedBlock{
who: tt.peerID,
header: tt.blockAnnounceHeader,
})
@@ -363,10 +445,10 @@ func TestChainSync_BootstrapSync_SuccessfulSync_WithOneWorker(t *testing.T) {
mockStorageState := NewMockStorageState(ctrl)
mockImportHandler := NewMockBlockImportHandler(ctrl)
mockTelemetry := NewMockTelemetry(ctrl)

EclesioMeloJunior marked this conversation as resolved.
Show resolved Hide resolved
const announceBlock = false
// setup mocks for new synced blocks that doesn't exists in our local database
ensureSuccessfulBlockImportFlow(t, mockedGenesisHeader, totalBlockResponse.BlockData, mockedBlockState,
mockBabeVerifier, mockStorageState, mockImportHandler, mockTelemetry)
mockBabeVerifier, mockStorageState, mockImportHandler, mockTelemetry, announceBlock)

// setup a chain sync which holds in its peer view map
// 3 peers, each one announce block X as its best block number.
@@ -422,10 +504,11 @@ func TestChainSync_BootstrapSync_SuccessfulSync_WithTwoWorkers(t *testing.T) {
worker1Response := &network.BlockResponseMessage{
BlockData: blockResponse.BlockData[:128],
}
const announceBlock = false
// the first peer will respond the from the block 1 to 128 so the ensureBlockImportFlow
// will setup the expectations starting from the genesis header until block 128
ensureSuccessfulBlockImportFlow(t, mockedGenesisHeader, worker1Response.BlockData, mockBlockState,
mockBabeVerifier, mockStorageState, mockImportHandler, mockTelemetry)
mockBabeVerifier, mockStorageState, mockImportHandler, mockTelemetry, announceBlock)

worker2Response := &network.BlockResponseMessage{
BlockData: blockResponse.BlockData[128:],
@@ -434,7 +517,7 @@ func TestChainSync_BootstrapSync_SuccessfulSync_WithTwoWorkers(t *testing.T) {
// will setup the expectations starting from block 128, from previous worker, until block 256
parent := worker1Response.BlockData[127]
ensureSuccessfulBlockImportFlow(t, parent.Header, worker2Response.BlockData, mockBlockState,
mockBabeVerifier, mockStorageState, mockImportHandler, mockTelemetry)
mockBabeVerifier, mockStorageState, mockImportHandler, mockTelemetry, announceBlock)

// we use gomock.Any since I cannot guarantee which peer picks which request
// but the first call to DoBlockRequest will return the first set and the second
@@ -496,15 +579,17 @@ func TestChainSync_BootstrapSync_SuccessfulSync_WithOneWorker_Failing(t *testing
// this test expects two workers responding each request with 128 blocks which means
// we should import 256 blocks in total
blockResponse := createSuccesfullBlockResponse(t, mockedGenesisHeader.Hash(), 1, 256)
const announceBlock = false

// here we split the whole set in two parts each one will be the "response" for each peer
worker1Response := &network.BlockResponseMessage{
BlockData: blockResponse.BlockData[:128],
}

// the first peer will respond the from the block 1 to 128 so the ensureBlockImportFlow
// will setup the expectations starting from the genesis header until block 128
ensureSuccessfulBlockImportFlow(t, mockedGenesisHeader, worker1Response.BlockData, mockBlockState,
mockBabeVerifier, mockStorageState, mockImportHandler, mockTelemetry)
mockBabeVerifier, mockStorageState, mockImportHandler, mockTelemetry, announceBlock)

worker2Response := &network.BlockResponseMessage{
BlockData: blockResponse.BlockData[128:],
@@ -513,7 +598,7 @@ func TestChainSync_BootstrapSync_SuccessfulSync_WithOneWorker_Failing(t *testing
// will setup the expectations starting from block 128, from previous worker, until block 256
parent := worker1Response.BlockData[127]
ensureSuccessfulBlockImportFlow(t, parent.Header, worker2Response.BlockData, mockBlockState,
mockBabeVerifier, mockStorageState, mockImportHandler, mockTelemetry)
mockBabeVerifier, mockStorageState, mockImportHandler, mockTelemetry, announceBlock)

// we use gomock.Any since I cannot guarantee which peer picks which request
// but the first call to DoBlockRequest will return the first set and the second
@@ -627,7 +712,7 @@ func createSuccesfullBlockResponse(_ *testing.T, genesisHash common.Hash,
func ensureSuccessfulBlockImportFlow(t *testing.T, parentHeader *types.Header,
blocksReceived []*types.BlockData, mockBlockState *MockBlockState,
mockBabeVerifier *MockBabeVerifier, mockStorageState *MockStorageState,
mockImportHandler *MockBlockImportHandler, mockTelemetry *MockTelemetry) {
mockImportHandler *MockBlockImportHandler, mockTelemetry *MockTelemetry, announceBlock bool) {
t.Helper()

mockBlockState.EXPECT().HasHeader(parentHeader.Hash()).Return(true, nil)
@@ -667,7 +752,7 @@ func ensureSuccessfulBlockImportFlow(t *testing.T, parentHeader *types.Header,
mockRuntimeInstance.EXPECT().ExecuteBlock(expectedBlock).
Return(nil, nil)

mockImportHandler.EXPECT().HandleBlockImport(expectedBlock, emptyTrieState, false).
mockImportHandler.EXPECT().HandleBlockImport(expectedBlock, emptyTrieState, announceBlock).
Return(nil)

blockHash := blockData.Header.Hash()
12 changes: 6 additions & 6 deletions dot/sync/mock_chain_sync_test.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion dot/sync/syncer.go
Original file line number Diff line number Diff line change
@@ -154,7 +154,7 @@ func (s *Service) HandleBlockAnnounce(from peer.ID, msg *network.BlockAnnounceMe
// that is also has the chain up until and including that block.
// this may not be a valid assumption, but perhaps we can assume that
// it is likely they will receive this block and its ancestors before us.
return s.chainSync.onImportBlock(announcedBlock{
return s.chainSync.onBlockAnnounce(announcedBlock{
who: from,
header: blockAnnounceHeader,
})
2 changes: 1 addition & 1 deletion dot/sync/syncer_test.go
Original file line number Diff line number Diff line change
@@ -239,7 +239,7 @@ func TestService_HandleBlockAnnounce(t *testing.T) {
header: block2AnnounceHeader,
}

chainSyncMock.EXPECT().onImportBlock(expectedAnnouncedBlock).Return(nil)
chainSyncMock.EXPECT().onBlockAnnounce(expectedAnnouncedBlock).Return(nil)

return &Service{
blockState: blockState,