Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[sovereign] Send bitmap outgoing operations #6703

Open
wants to merge 7 commits into
base: feat/sovereign-cross-validators-changes
Choose a base branch
from
Open

[sovereign] Send bitmap outgoing operations #6703

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
1b65254
FEAT: Comm placeholder for future code
mariusmihaic Jan 13, 2025
10941f3
FEAT: Update core go mod with new proto outgoing data
mariusmihaic Jan 13, 2025
33c4904
FEAT: Simple version to send directly bitmap + epoch without signatures
mariusmihaic Jan 14, 2025
7a18b9d
CLN: Version not to sign bitmap + epoch on bridge data
mariusmihaic Jan 15, 2025
d19d1f9
FIX: Tests in sovereignSubRoundEnd_test.go
mariusmihaic Jan 15, 2025
316d228
FEAT: Extend tests in sovereignSubRoundEnd_test.go
mariusmihaic Jan 15, 2025
8f3f1eb
FIX: bitmap for updateBridgeDataWithSignatures in sovereignSubRoundEn…
mariusmihaic Jan 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion cmd/sovereignnode/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ go 1.20

require (
github.com/google/gops v0.3.18
github.com/multiversx/mx-chain-core-go v1.2.24-0.20241218131403-473853d0b54a
github.com/multiversx/mx-chain-core-go v1.2.24-0.20250113143225-72ba2e508401
github.com/multiversx/mx-chain-go v1.8.4
github.com/multiversx/mx-chain-logger-go v1.0.15
github.com/multiversx/mx-chain-sovereign-bridge-go v0.0.0-20240116102202-4cf6fbbd95a3
Expand Down
4 changes: 2 additions & 2 deletions cmd/sovereignnode/go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -385,8 +385,8 @@ github.com/multiversx/concurrent-map v0.1.4 h1:hdnbM8VE4b0KYJaGY5yJS2aNIW9TFFsUY
github.com/multiversx/concurrent-map v0.1.4/go.mod h1:8cWFRJDOrWHOTNSqgYCUvwT7c7eFQ4U2vKMOp4A/9+o=
github.com/multiversx/mx-chain-communication-go v1.1.1 h1:y4DoQeQOJTaSUsRzczQFazf8JYQmInddypApqA3AkwM=
github.com/multiversx/mx-chain-communication-go v1.1.1/go.mod h1:WK6bP4pGEHGDDna/AYRIMtl6G9OA0NByI1Lw8PmOnRM=
github.com/multiversx/mx-chain-core-go v1.2.24-0.20241218131403-473853d0b54a h1:0w+YAzjGWLV+U3ERmu1TnUUgwjj+3kAbTL4S+vCUa2M=
github.com/multiversx/mx-chain-core-go v1.2.24-0.20241218131403-473853d0b54a/go.mod h1:P/YBoFnt25XUaCQ7Q/SD15vhnc9yV5JDhHxyFO9P8Z0=
github.com/multiversx/mx-chain-core-go v1.2.24-0.20250113143225-72ba2e508401 h1:jMSqlYlzHwbbjYerMGNZtZsm4eZshH3uXW9N8iNbHD0=
github.com/multiversx/mx-chain-core-go v1.2.24-0.20250113143225-72ba2e508401/go.mod h1:P/YBoFnt25XUaCQ7Q/SD15vhnc9yV5JDhHxyFO9P8Z0=
github.com/multiversx/mx-chain-crypto-go v1.2.12 h1:zWip7rpUS4CGthJxfKn5MZfMfYPjVjIiCID6uX5BSOk=
github.com/multiversx/mx-chain-crypto-go v1.2.12/go.mod h1:HzcPpCm1zanNct/6h2rIh+MFrlXbjA5C8+uMyXj3LI4=
github.com/multiversx/mx-chain-es-indexer-go v1.7.11-0.20241211094055-23de51462ca4 h1:IfjK6IQy65eMWr/FEv3CMUz13ZqH9sWLnKv85/AoayM=
Expand Down
2 changes: 1 addition & 1 deletion cmd/sovereignnode/systemTestDemo/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ go 1.20

require (
github.com/multiversx/mx-chain-communication-go v1.1.1
github.com/multiversx/mx-chain-core-go v1.2.24-0.20241111110328-b8a9c02d5583
github.com/multiversx/mx-chain-core-go v1.2.24-0.20250113143225-72ba2e508401
github.com/multiversx/mx-chain-crypto-go v1.2.12
github.com/multiversx/mx-chain-logger-go v1.0.15
github.com/multiversx/mx-chain-sovereign-bridge-go v0.0.0-20240116102202-4cf6fbbd95a3
Expand Down
4 changes: 2 additions & 2 deletions cmd/sovereignnode/systemTestDemo/go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,8 +100,8 @@ github.com/mr-tron/base58 v1.2.0 h1:T/HDJBh4ZCPbU39/+c3rRvE0uKBQlU27+QI8LJ4t64o=
github.com/mr-tron/base58 v1.2.0/go.mod h1:BinMc/sQntlIE1frQmRFPUoPA1Zkr8VRgBdjWI2mNwc=
github.com/multiversx/mx-chain-communication-go v1.1.1 h1:y4DoQeQOJTaSUsRzczQFazf8JYQmInddypApqA3AkwM=
github.com/multiversx/mx-chain-communication-go v1.1.1/go.mod h1:WK6bP4pGEHGDDna/AYRIMtl6G9OA0NByI1Lw8PmOnRM=
github.com/multiversx/mx-chain-core-go v1.2.24-0.20241111110328-b8a9c02d5583 h1:EEYn25eF4DV1UTcCjOarSH1voitQ+YCnFmbDgJzLqB8=
github.com/multiversx/mx-chain-core-go v1.2.24-0.20241111110328-b8a9c02d5583/go.mod h1:P/YBoFnt25XUaCQ7Q/SD15vhnc9yV5JDhHxyFO9P8Z0=
github.com/multiversx/mx-chain-core-go v1.2.24-0.20250113143225-72ba2e508401 h1:jMSqlYlzHwbbjYerMGNZtZsm4eZshH3uXW9N8iNbHD0=
github.com/multiversx/mx-chain-core-go v1.2.24-0.20250113143225-72ba2e508401/go.mod h1:P/YBoFnt25XUaCQ7Q/SD15vhnc9yV5JDhHxyFO9P8Z0=
github.com/multiversx/mx-chain-crypto-go v1.2.12 h1:zWip7rpUS4CGthJxfKn5MZfMfYPjVjIiCID6uX5BSOk=
github.com/multiversx/mx-chain-crypto-go v1.2.12/go.mod h1:HzcPpCm1zanNct/6h2rIh+MFrlXbjA5C8+uMyXj3LI4=
github.com/multiversx/mx-chain-go v1.7.12 h1:cQ3g5sFZEcQmIRwi/wt+K/3d5nIwCMQRC1ZnJDRuRY4=
Expand Down
7 changes: 4 additions & 3 deletions consensus/spos/bls/sovereignSubRoundEnd.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ func (sr *sovereignSubRoundEnd) updateOutGoingPoolIfNeeded(cnsDta *consensus.Mes
"AggregatedSignatureOutGoingTxData", cnsDta.AggregatedSignatureOutGoingTxData,
)

_, err = sr.updateBridgeDataWithSignatures(outGoingMBHeader)
_, err = sr.updateBridgeDataWithSignatures(outGoingMBHeader, cnsDta.PubKeysBitmap)
if err != nil {
log.Error("sovereignSubRoundEnd.updateOutGoingPoolIfNeeded.updateBridgeDataWithSignatures", "error", err)
return err
Expand All @@ -113,7 +113,7 @@ func (sr *sovereignSubRoundEnd) doSovereignEndRoundJob(ctx context.Context) bool
return true
}

currBridgeData, err := sr.updateBridgeDataWithSignatures(outGoingMBHeader)
currBridgeData, err := sr.updateBridgeDataWithSignatures(outGoingMBHeader, sovHeader.GetPubKeysBitmap())
if err != nil {
log.Error("sovereignSubRoundEnd.doSovereignEndRoundJob.updateBridgeDataWithSignatures", "error", err)
return false
Expand Down Expand Up @@ -144,7 +144,7 @@ func (sr *sovereignSubRoundEnd) sendUnconfirmedOperationsIfFound(ctx context.Con
}

func (sr *sovereignSubRoundEnd) updateBridgeDataWithSignatures(
outGoingMBHeader data.OutGoingMiniBlockHeaderHandler,
outGoingMBHeader data.OutGoingMiniBlockHeaderHandler, pubKeysBitmap []byte,
) (*sovereign.BridgeOutGoingData, error) {
hash := outGoingMBHeader.GetOutGoingOperationsHash()
currBridgeData := sr.outGoingOperationsPool.Get(hash)
Expand All @@ -155,6 +155,7 @@ func (sr *sovereignSubRoundEnd) updateBridgeDataWithSignatures(

currBridgeData.LeaderSignature = outGoingMBHeader.GetLeaderSignatureOutGoingOperations()
currBridgeData.AggregatedSignature = outGoingMBHeader.GetAggregatedSignatureOutGoingOperations()
currBridgeData.PubKeysBitmap = pubKeysBitmap

sr.outGoingOperationsPool.Delete(hash)
sr.outGoingOperationsPool.Add(currBridgeData)
Expand Down
1 change: 1 addition & 0 deletions consensus/spos/bls/sovereignSubRoundEndOutGoingTxData.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import (

"github.com/multiversx/mx-chain-core-go/core/check"
"github.com/multiversx/mx-chain-core-go/data"

"github.com/multiversx/mx-chain-go/consensus"
"github.com/multiversx/mx-chain-go/consensus/spos"
"github.com/multiversx/mx-chain-go/errors"
Expand Down
15 changes: 14 additions & 1 deletion consensus/spos/bls/sovereignSubRoundEnd_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ import (
"github.com/multiversx/mx-chain-core-go/data"
"github.com/multiversx/mx-chain-core-go/data/block"
sovCore "github.com/multiversx/mx-chain-core-go/data/sovereign"
"github.com/stretchr/testify/require"

"github.com/multiversx/mx-chain-go/consensus"
"github.com/multiversx/mx-chain-go/consensus/mock"
"github.com/multiversx/mx-chain-go/consensus/spos"
Expand All @@ -18,7 +20,6 @@ import (
"github.com/multiversx/mx-chain-go/testscommon/enableEpochsHandlerMock"
"github.com/multiversx/mx-chain-go/testscommon/sovereign"
"github.com/multiversx/mx-chain-go/testscommon/statusHandler"
"github.com/stretchr/testify/require"
)

type sovEndRoundHandler interface {
Expand Down Expand Up @@ -175,6 +176,7 @@ func TestSovereignSubRoundEnd_DoEndJobByLeader(t *testing.T) {
outGoingOpData := []byte("bridgeOp")
aggregatedSig := []byte("aggregatedSig")
leaderSig := []byte("leaderSig")
pubKeysBitmap := []byte{0x1, 0x0}
getCallCt := 0
wasResetTimerCalled := false
wg := sync.WaitGroup{}
Expand Down Expand Up @@ -218,6 +220,7 @@ func TestSovereignSubRoundEnd_DoEndJobByLeader(t *testing.T) {
},
AggregatedSignature: aggregatedSig,
LeaderSignature: leaderSig,
PubKeysBitmap: pubKeysBitmap,
}, data)
},
ResetTimerCalled: func(hashes [][]byte) {
Expand Down Expand Up @@ -251,6 +254,7 @@ func TestSovereignSubRoundEnd_DoEndJobByLeader(t *testing.T) {
},
LeaderSignature: leaderSig,
AggregatedSignature: aggregatedSig,
PubKeysBitmap: pubKeysBitmap,
},
},
}, data)
Expand Down Expand Up @@ -298,6 +302,7 @@ func TestSovereignSubRoundEnd_DoEndJobByLeader(t *testing.T) {
},
AggregatedSignature: aggregatedSig,
LeaderSignature: leaderSig,
Epoch: 4,
}

unconfirmedBridgeOutGoingData := &sovCore.BridgeOutGoingData{
Expand All @@ -308,6 +313,7 @@ func TestSovereignSubRoundEnd_DoEndJobByLeader(t *testing.T) {
Data: []byte("bridgeOp2"),
},
},
Epoch: 3,
}

wasResetTimerCalled := false
Expand Down Expand Up @@ -352,6 +358,7 @@ func TestSovereignSubRoundEnd_DoEndJobByLeader(t *testing.T) {
sovHdr := &block.SovereignChainHeader{
Header: &block.Header{
Nonce: 4,
Epoch: 4,
},
OutGoingMiniBlockHeader: &block.OutGoingMiniBlockHeader{
OutGoingOperationsHash: outGoingDataHash,
Expand Down Expand Up @@ -436,6 +443,7 @@ func TestSovereignSubRoundEnd_DoEndJobByLeader(t *testing.T) {
t.Run("leader keeps resending previous unconfirmed operations, should keep signatures in pool", func(t *testing.T) {
t.Parallel()

epoch := uint32(4)
unconfirmedBridgeOutGoingData := &sovCore.BridgeOutGoingData{
Hash: []byte("hashOfHashes"),
OutGoingOperations: []*sovCore.OutGoingOperation{
Expand All @@ -444,12 +452,14 @@ func TestSovereignSubRoundEnd_DoEndJobByLeader(t *testing.T) {
Data: []byte("bridgeOp1"),
},
},
Epoch: epoch,
}

expiryTime := time.Millisecond * 100
pool := sovereignBlock.NewOutGoingOperationPool(expiryTime)
pool.Add(unconfirmedBridgeOutGoingData)

pubKeysBitmap := []byte{0x1, 0x0}
sendDataCalledCt := 0
wasDataSent := false
currCtx := context.Background()
Expand All @@ -467,6 +477,8 @@ func TestSovereignSubRoundEnd_DoEndJobByLeader(t *testing.T) {
},
AggregatedSignature: []byte("aggregatedSig"),
LeaderSignature: []byte("leaderSig"),
PubKeysBitmap: pubKeysBitmap,
Epoch: epoch,
},
},
}
Expand All @@ -488,6 +500,7 @@ func TestSovereignSubRoundEnd_DoEndJobByLeader(t *testing.T) {
sovHdr := &block.SovereignChainHeader{
Header: &block.Header{
Nonce: 4,
Epoch: epoch,
},
OutGoingMiniBlockHeader: &block.OutGoingMiniBlockHeader{
Hash: []byte("hashOfHashes"),
Expand Down
1 change: 1 addition & 0 deletions consensus/spos/bls/sovereignSubRoundSignatureOutGoingTxData.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import (

"github.com/multiversx/mx-chain-core-go/core/check"
"github.com/multiversx/mx-chain-core-go/data"

"github.com/multiversx/mx-chain-go/consensus"
"github.com/multiversx/mx-chain-go/consensus/spos"
"github.com/multiversx/mx-chain-go/errors"
Expand Down
2 changes: 2 additions & 0 deletions dataRetriever/dataPool/sovereign/outGoingOperationPool.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,8 @@ func (op *outGoingOperationsPool) Add(data *sovereign.BridgeOutGoingData) {
"hash", data.Hash,
"aggregated sig", data.AggregatedSignature,
"leader sig", data.LeaderSignature,
"pub keys bitmap", data.PubKeysBitmap,
"epoch", data.Epoch,
)

hashStr := string(data.Hash)
Expand Down
1 change: 1 addition & 0 deletions factory/core/coreComponentsHandler.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import (
"github.com/multiversx/mx-chain-core-go/data/typeConverters"
"github.com/multiversx/mx-chain-core-go/hashing"
"github.com/multiversx/mx-chain-core-go/marshal"

"github.com/multiversx/mx-chain-go/common"
"github.com/multiversx/mx-chain-go/consensus"
"github.com/multiversx/mx-chain-go/errors"
Expand Down
2 changes: 1 addition & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ require (
github.com/klauspost/cpuid/v2 v2.2.5
github.com/mitchellh/mapstructure v1.5.0
github.com/multiversx/mx-chain-communication-go v1.1.1
github.com/multiversx/mx-chain-core-go v1.2.24-0.20241218131403-473853d0b54a
github.com/multiversx/mx-chain-core-go v1.2.24-0.20250113143225-72ba2e508401
github.com/multiversx/mx-chain-crypto-go v1.2.12
github.com/multiversx/mx-chain-es-indexer-go v1.7.11-0.20241211094055-23de51462ca4
github.com/multiversx/mx-chain-logger-go v1.0.15
Expand Down
4 changes: 2 additions & 2 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -387,8 +387,8 @@ github.com/multiversx/concurrent-map v0.1.4 h1:hdnbM8VE4b0KYJaGY5yJS2aNIW9TFFsUY
github.com/multiversx/concurrent-map v0.1.4/go.mod h1:8cWFRJDOrWHOTNSqgYCUvwT7c7eFQ4U2vKMOp4A/9+o=
github.com/multiversx/mx-chain-communication-go v1.1.1 h1:y4DoQeQOJTaSUsRzczQFazf8JYQmInddypApqA3AkwM=
github.com/multiversx/mx-chain-communication-go v1.1.1/go.mod h1:WK6bP4pGEHGDDna/AYRIMtl6G9OA0NByI1Lw8PmOnRM=
github.com/multiversx/mx-chain-core-go v1.2.24-0.20241218131403-473853d0b54a h1:0w+YAzjGWLV+U3ERmu1TnUUgwjj+3kAbTL4S+vCUa2M=
github.com/multiversx/mx-chain-core-go v1.2.24-0.20241218131403-473853d0b54a/go.mod h1:P/YBoFnt25XUaCQ7Q/SD15vhnc9yV5JDhHxyFO9P8Z0=
github.com/multiversx/mx-chain-core-go v1.2.24-0.20250113143225-72ba2e508401 h1:jMSqlYlzHwbbjYerMGNZtZsm4eZshH3uXW9N8iNbHD0=
github.com/multiversx/mx-chain-core-go v1.2.24-0.20250113143225-72ba2e508401/go.mod h1:P/YBoFnt25XUaCQ7Q/SD15vhnc9yV5JDhHxyFO9P8Z0=
github.com/multiversx/mx-chain-crypto-go v1.2.12 h1:zWip7rpUS4CGthJxfKn5MZfMfYPjVjIiCID6uX5BSOk=
github.com/multiversx/mx-chain-crypto-go v1.2.12/go.mod h1:HzcPpCm1zanNct/6h2rIh+MFrlXbjA5C8+uMyXj3LI4=
github.com/multiversx/mx-chain-es-indexer-go v1.7.11-0.20241211094055-23de51462ca4 h1:IfjK6IQy65eMWr/FEv3CMUz13ZqH9sWLnKv85/AoayM=
Expand Down
6 changes: 4 additions & 2 deletions process/block/sovereignChainBlock.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1401,11 +1401,11 @@ func (scbp *sovereignChainBlockProcessor) createAndSetOutGoingMiniBlock(headerHa
return nil
}

outGoingMb, outGoingOperationsHash := scbp.createOutGoingMiniBlockData(outGoingOperations)
outGoingMb, outGoingOperationsHash := scbp.createOutGoingMiniBlockData(headerHandler, outGoingOperations)
return scbp.setOutGoingMiniBlock(headerHandler, createdBlockBody, outGoingMb, outGoingOperationsHash)
}

func (scbp *sovereignChainBlockProcessor) createOutGoingMiniBlockData(outGoingOperations [][]byte) (*block.MiniBlock, []byte) {
func (scbp *sovereignChainBlockProcessor) createOutGoingMiniBlockData(headerHandler data.HeaderHandler, outGoingOperations [][]byte) (*block.MiniBlock, []byte) {
outGoingOpHashes := make([][]byte, len(outGoingOperations))
aggregatedOutGoingOperations := make([]byte, 0)
outGoingOperationsData := make([]*sovCore.OutGoingOperation, 0)
Expand All @@ -1428,6 +1428,8 @@ func (scbp *sovereignChainBlockProcessor) createOutGoingMiniBlockData(outGoingOp
scbp.outGoingOperationsPool.Add(&sovCore.BridgeOutGoingData{
Hash: outGoingOperationsHash,
OutGoingOperations: outGoingOperationsData,
PubKeysBitmap: headerHandler.GetPubKeysBitmap(),
Epoch: headerHandler.GetEpoch(),
})

return &block.MiniBlock{
Expand Down
15 changes: 14 additions & 1 deletion process/block/sovereignChainBlock_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -265,7 +265,9 @@ func TestSovereignChainBlockProcessor_createAndSetOutGoingMiniBlock(t *testing.T
},
}

epoch := uint32(4)
poolAddCt := 0
pubKeysBitmap := []byte("pubKeysBitmap")
outGoingOperationsPool := &sovereign.OutGoingOperationsPoolMock{
AddCalled: func(data *sovereignCore.BridgeOutGoingData) {
defer func() {
Expand All @@ -286,6 +288,8 @@ func TestSovereignChainBlockProcessor_createAndSetOutGoingMiniBlock(t *testing.T
Data: bridgeOp2,
},
},
PubKeysBitmap: pubKeysBitmap,
Epoch: epoch,
}, data)
default:
require.Fail(t, "should not add in pool any other operation")
Expand All @@ -308,7 +312,12 @@ func TestSovereignChainBlockProcessor_createAndSetOutGoingMiniBlock(t *testing.T
SCToProtocol: &mock.SCToProtocolStub{},
})

sovChainHdr := &block.SovereignChainHeader{}
sovChainHdr := &block.SovereignChainHeader{
Header: &block.Header{
Epoch: epoch,
PubKeysBitmap: pubKeysBitmap,
},
}
processedMb := &block.MiniBlock{
ReceiverShardID: core.SovereignChainShardId,
SenderShardID: core.MainChainShardId,
Expand All @@ -335,6 +344,10 @@ func TestSovereignChainBlockProcessor_createAndSetOutGoingMiniBlock(t *testing.T
require.Nil(t, err)

expectedSovChainHeader := &block.SovereignChainHeader{
Header: &block.Header{
Epoch: epoch,
PubKeysBitmap: pubKeysBitmap,
},
OutGoingMiniBlockHeader: &block.OutGoingMiniBlockHeader{
Hash: expectedOutGoingMbHash,
OutGoingOperationsHash: bridgeOpsHash,
Expand Down
1 change: 1 addition & 0 deletions process/headerCheck/sovereignHeaderSignatureVerifier.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import (
"github.com/multiversx/mx-chain-core-go/core/check"
"github.com/multiversx/mx-chain-core-go/data"
crypto "github.com/multiversx/mx-chain-crypto-go"

"github.com/multiversx/mx-chain-go/errors"
"github.com/multiversx/mx-chain-go/process"
)
Expand Down
Loading