Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AggLayer integration #71

Closed
wants to merge 64 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
6e8ad9e
fix migration batch wip field. fix sequence when no wip batch (#3126)
agnusmor Jan 23, 2024
22b2651
fix Leaves smtproof (#3127)
ARR552 Jan 23, 2024
1962ca1
Add l2block log dump when storing error. Refactor ProcessBatchV2 logs…
agnusmor Jan 24, 2024
c3dddf7
fix panic etherman (#3133)
ARR552 Jan 24, 2024
8e1e5e9
Fix GetL2Hash function. Use tx L2 hash returned by the executor when …
agnusmor Jan 24, 2024
19d493e
remove l2 tx hash computation and use data stored in the state return…
tclemos Jan 25, 2024
8124e52
etrog: fix permissionless errors (#3140)
joanestebanr Jan 25, 2024
895c1e0
Fix missing batch bookmark (#3122)
ToniRamirezM Jan 25, 2024
6feb7b8
fix wipL2Block deltaTimestamp (#3142)
agnusmor Jan 25, 2024
646603e
ensure tx order in stream (#3144)
ToniRamirezM Jan 25, 2024
4ddba20
fix wipL2Block stateRoot (#3145)
agnusmor Jan 25, 2024
1825164
fix panic NewStackTrie (#3146)
ARR552 Jan 25, 2024
bc51813
fix sync initalization for etrog (#3147)
tclemos Jan 25, 2024
4dc6258
fix wipL2block imStateRoot (#3148)
agnusmor Jan 26, 2024
86fcb5b
fix tracer to work by block stateroot instead of by tx (#3150)
tclemos Jan 26, 2024
75a957f
update prover image (#3151)
ToniRamirezM Jan 26, 2024
f69f0f9
if a trusted batch is empty and WIP just create the entry in state.ba…
joanestebanr Jan 26, 2024
65a507e
fix trace for tx index greater than 0 (#3153)
tclemos Jan 26, 2024
8f79b21
add cardona.zip (#3154)
joanestebanr Jan 26, 2024
c100d40
Fix default value when creating transaction.used_sha256_hashes field …
agnusmor Jan 26, 2024
1a8dd7d
add cardona testnet (#2909) (#3155)
joanestebanr Jan 26, 2024
f3b86d4
new fields in stream (#3149)
ToniRamirezM Jan 26, 2024
b47986a
set stateManagerPurge to false in Cardona (#3158)
joanestebanr Jan 26, 2024
eea3833
synchronizer: fix case emtpy batch and unittest (#3159)
joanestebanr Jan 27, 2024
7497d07
update prover image to v4.0.1 (#3160)
agnusmor Jan 27, 2024
12022c2
update prover image to v4.0.2 (#3162)
agnusmor Jan 27, 2024
a296484
Add a note in deployment instruction to restart rpc after forkId 7 (#…
joanestebanr Jan 27, 2024
5dfe003
change position of point 10
agnusmor Jan 27, 2024
479ba46
add point 11
agnusmor Jan 27, 2024
9e2a95d
add lines
agnusmor Jan 27, 2024
b216df0
add breakline
agnusmor Jan 27, 2024
f5bc983
add lines
agnusmor Jan 27, 2024
87bb731
add line
agnusmor Jan 27, 2024
18a75ff
Sync old network compatibility (#3166)
ARR552 Jan 30, 2024
f5346fa
synchronizer: unittest and remove pool_reorg (#3111) (#3168)
joanestebanr Jan 30, 2024
efbd360
fix parentHash when storing a new L2 block (#3171)
agnusmor Jan 30, 2024
6b49ba8
Fix/rom error (#3177)
ARR552 Jan 31, 2024
b55cf66
Cherrypick/#3179 geth version v1.13.11 (#3180)
ARR552 Jan 31, 2024
291ac85
Add sanity check: last L2 block timestamp in the sequence is also L1B…
agnusmor Jan 31, 2024
5583622
synchronizer: update fromTrusted fix cache update (#3169)
joanestebanr Feb 1, 2024
a110d2e
Feature/3173 sync trusted state in networks previous to ETROG (#3175)
joanestebanr Feb 1, 2024
10aaffb
Fix/3164 eth syncing dont return expected values (#3165)
joanestebanr Feb 1, 2024
f3dfbf2
fix im state root in stream (#3192)
ToniRamirezM Feb 2, 2024
6f81efa
sync: fix getting URL from contract in compatibility mode, and set se…
joanestebanr Feb 2, 2024
175e0f3
Fix isSynced function (#3188)
agnusmor Feb 2, 2024
3567c84
fix l2 block gas price to use value returned by the executor (#3186) …
tclemos Feb 2, 2024
284dda3
allow rpc to access the ForkID directly from DB (#3196)
tclemos Feb 2, 2024
95465ca
allow rpc to access the ForkID directly from DB (#3197)
tclemos Feb 3, 2024
112bdd0
Fix/3200 l1inforoot error on reorg (#3201)
joanestebanr Feb 5, 2024
a4059e6
Fix/3203 panic if batch process fail and must be closed (#3206)
joanestebanr Feb 5, 2024
d439c9f
Remove L1SynchronizationMode parameter from environments config files…
agnusmor Feb 5, 2024
78ba11d
allow GetL2TxHashByTxHash to return null for txs prior to Etrog (#3209)
tclemos Feb 5, 2024
d85d135
Fix L2 block gas limit to 2**50 (1125899906842624) (#3219)
agnusmor Feb 6, 2024
c29d96c
change 2^64-1 for an esmitation of blocks (#3220)
joanestebanr Feb 6, 2024
e616ee5
removed restart RPC on deploy instructions (#3221)
joanestebanr Feb 6, 2024
18b38d7
Cherry pick/3222 (#3224)
ToniRamirezM Feb 6, 2024
bab6187
add retries in is Synced function when last virtual batch is greater …
agnusmor Feb 7, 2024
6044b9b
wip
rachit77 Feb 7, 2024
e88984a
wip
rachit77 Feb 7, 2024
514deb9
update mocks
rachit77 Feb 7, 2024
df3f71b
wip
rachit77 Feb 7, 2024
1ed2128
modified sequencesender interfaces signatures
rachit77 Feb 7, 2024
a5b8cd5
linter fix
rachit77 Feb 7, 2024
6fbef6a
AggLayer integration
vcastellm Feb 7, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
157 changes: 137 additions & 20 deletions aggregator/aggregator.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ package aggregator

import (
"context"
"crypto/ecdsa"
"encoding/json"
"errors"
"fmt"
Expand All @@ -13,6 +14,9 @@ import (
"time"
"unicode"

"github.com/0xPolygon/agglayer/client"
agglayerTypes "github.com/0xPolygon/agglayer/rpc/types"
"github.com/0xPolygon/agglayer/tx"
"github.com/0xPolygonHermez/zkevm-node/aggregator/metrics"
"github.com/0xPolygonHermez/zkevm-node/aggregator/prover"
"github.com/0xPolygonHermez/zkevm-node/config/types"
Expand Down Expand Up @@ -65,6 +69,9 @@ type Aggregator struct {
srv *grpc.Server
ctx context.Context
exit context.CancelFunc

AggLayerClient client.ClientInterface
sequencerPrivateKey *ecdsa.PrivateKey
}

// New creates a new aggregator.
Expand All @@ -73,6 +80,8 @@ func New(
stateInterface stateInterface,
ethTxManager ethTxManager,
etherman etherman,
agglayerClient client.ClientInterface,
sequencerPrivateKey *ecdsa.PrivateKey,
) (Aggregator, error) {
var profitabilityChecker aggregatorTxProfitabilityChecker
switch cfg.TxProfitabilityCheckerType {
Expand All @@ -94,6 +103,9 @@ func New(
TimeCleanupLockedProofs: cfg.CleanupLockedProofsInterval,

finalProof: make(chan finalProofMsg),

AggLayerClient: agglayerClient,
sequencerPrivateKey: sequencerPrivateKey,
}

return a, nil
Expand Down Expand Up @@ -267,34 +279,139 @@ func (a *Aggregator) sendFinalProof() {

log.Infof("Final proof inputs: NewLocalExitRoot [%#x], NewStateRoot [%#x]", inputs.NewLocalExitRoot, inputs.NewStateRoot)

// add batch verification to be monitored
sender := common.HexToAddress(a.cfg.SenderAddress)
to, data, err := a.Ethman.BuildTrustedVerifyBatchesTxData(proof.BatchNumber-1, proof.BatchNumberFinal, &inputs, sender)
if err != nil {
log.Errorf("Error estimating batch verification to add to eth tx manager: %v", err)
a.handleFailureToAddVerifyBatchToBeMonitored(ctx, proof)
continue
}
monitoredTxID := buildMonitoredTxID(proof.BatchNumber, proof.BatchNumberFinal)
err = a.EthTxManager.Add(ctx, ethTxManagerOwner, monitoredTxID, sender, to, nil, data, a.cfg.GasOffset, nil)
if err != nil {
mTxLogger := ethtxmanager.CreateLogger(ethTxManagerOwner, monitoredTxID, sender, to)
mTxLogger.Errorf("Error to add batch verification tx to eth tx manager: %v", err)
a.handleFailureToAddVerifyBatchToBeMonitored(ctx, proof)
continue
switch a.cfg.SettlementBackend {
case AggLayer:
if success := a.settleWithAggLayer(ctx, proof, inputs); !success {
continue
}
default:
if success := a.settleDirect(ctx, proof, inputs); !success {
continue
}
}

// process monitored batch verifications before starting a next cycle
a.EthTxManager.ProcessPendingMonitoredTxs(ctx, ethTxManagerOwner, func(result ethtxmanager.MonitoredTxResult, dbTx pgx.Tx) {
a.handleMonitoredTxResult(result)
}, nil)

a.resetVerifyProofTime()
a.endProofVerification()
}
}
}

func (a *Aggregator) settleDirect(
ctx context.Context,
proof *state.Proof,
inputs ethmanTypes.FinalProofInputs,
) (success bool) {
// add batch verification to be monitored
sender := common.HexToAddress(a.cfg.SenderAddress)

to, data, err := a.Ethman.BuildTrustedVerifyBatchesTxData(
proof.BatchNumber-1,
proof.BatchNumberFinal,
&inputs,
sender,
)
if err != nil {
log.Errorf("Error estimating batch verification to add to eth tx manager: %v", err)
a.handleFailureToAddVerifyBatchToBeMonitored(ctx, proof)

return false
}

monitoredTxID := buildMonitoredTxID(proof.BatchNumber, proof.BatchNumberFinal)
err = a.EthTxManager.Add(
ctx,
ethTxManagerOwner,
monitoredTxID,
sender,
to,
nil,
data,
a.cfg.GasOffset,
nil,
)
if err != nil {
mTxLogger := ethtxmanager.CreateLogger(ethTxManagerOwner, monitoredTxID, sender, to)
mTxLogger.Errorf("Error to add batch verification tx to eth tx manager: %v", err)
a.handleFailureToAddVerifyBatchToBeMonitored(ctx, proof)

return false
}

// process monitored batch verifications before starting a next cycle
a.EthTxManager.ProcessPendingMonitoredTxs(
ctx,
ethTxManagerOwner,
func(result ethtxmanager.MonitoredTxResult, dbTx pgx.Tx) {
a.handleMonitoredTxResult(result)
},
nil,
)

return true
}

func (a *Aggregator) settleWithAggLayer(
ctx context.Context,
proof *state.Proof,
inputs ethmanTypes.FinalProofInputs,
) (success bool) {
proofStrNo0x := strings.TrimPrefix(inputs.FinalProof.Proof, "0x")
proofBytes := common.Hex2Bytes(proofStrNo0x)
tx := tx.Tx{
LastVerifiedBatch: agglayerTypes.ArgUint64(proof.BatchNumber - 1),
NewVerifiedBatch: agglayerTypes.ArgUint64(proof.BatchNumberFinal),
ZKP: tx.ZKP{
NewStateRoot: common.BytesToHash(inputs.NewStateRoot),
NewLocalExitRoot: common.BytesToHash(inputs.NewLocalExitRoot),
Proof: agglayerTypes.ArgBytes(proofBytes),
},
RollupID: a.Ethman.GetRollupId(),
}
signedTx, err := tx.Sign(a.sequencerPrivateKey)

if err != nil {
log.Errorf("failed to sign tx: %v", err)
a.handleFailureToSendToAggLayer(ctx, proof)

return false
}

log.Debug("final proof signedTx: ", signedTx.Tx.ZKP.Proof.Hex())
txHash, err := a.AggLayerClient.SendTx(*signedTx)
if err != nil {
log.Errorf("failed to send tx to the interop: %v", err)
a.handleFailureToSendToAggLayer(ctx, proof)

return false
}

log.Infof("tx %s sent to agglayer, waiting to be mined", txHash.Hex())
log.Debugf("Timeout set to %f seconds", a.cfg.AggLayerTxTimeout.Duration.Seconds())
waitCtx, cancelFunc := context.WithDeadline(ctx, time.Now().Add(a.cfg.AggLayerTxTimeout.Duration))
defer cancelFunc()
if err := a.AggLayerClient.WaitTxToBeMined(txHash, waitCtx); err != nil {
log.Errorf("interop didn't mine the tx: %v", err)
a.handleFailureToSendToAggLayer(ctx, proof)

return false
}

// TODO: wait for synchronizer to catch up
return true
}

func (a *Aggregator) handleFailureToSendToAggLayer(ctx context.Context, proof *state.Proof) {
log := log.WithFields("proofId", proof.ProofID, "batches", fmt.Sprintf("%d-%d", proof.BatchNumber, proof.BatchNumberFinal))
proof.GeneratingSince = nil

err := a.State.UpdateGeneratedProof(ctx, proof, nil)
if err != nil {
log.Errorf("Failed updating proof state (false): %v", err)
}

a.endProofVerification()
}

func (a *Aggregator) handleFailureToAddVerifyBatchToBeMonitored(ctx context.Context, proof *state.Proof) {
log := log.WithFields("proofId", proof.ProofID, "batches", fmt.Sprintf("%d-%d", proof.BatchNumber, proof.BatchNumberFinal))
proof.GeneratingSince = nil
Expand Down
10 changes: 5 additions & 5 deletions aggregator/aggregator_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -190,7 +190,7 @@ func TestSendFinalProof(t *testing.T) {
stateMock := mocks.NewStateMock(t)
ethTxManager := mocks.NewEthTxManager(t)
etherman := mocks.NewEtherman(t)
a, err := New(cfg, stateMock, ethTxManager, etherman)
a, err := New(cfg, stateMock, ethTxManager, etherman, nil, nil)
require.NoError(err)
a.ctx, a.exit = context.WithCancel(context.Background())
m := mox{
Expand Down Expand Up @@ -685,7 +685,7 @@ func TestTryAggregateProofs(t *testing.T) {
ethTxManager := mocks.NewEthTxManager(t)
etherman := mocks.NewEtherman(t)
proverMock := mocks.NewProverMock(t)
a, err := New(cfg, stateMock, ethTxManager, etherman)
a, err := New(cfg, stateMock, ethTxManager, etherman, nil, nil)
require.NoError(err)
aggregatorCtx := context.WithValue(context.Background(), "owner", "aggregator") //nolint:staticcheck
a.ctx, a.exit = context.WithCancel(aggregatorCtx)
Expand Down Expand Up @@ -1023,7 +1023,7 @@ func TestTryGenerateBatchProof(t *testing.T) {
ethTxManager := mocks.NewEthTxManager(t)
etherman := mocks.NewEtherman(t)
proverMock := mocks.NewProverMock(t)
a, err := New(cfg, stateMock, ethTxManager, etherman)
a, err := New(cfg, stateMock, ethTxManager, etherman, nil, nil)
require.NoError(err)
aggregatorCtx := context.WithValue(context.Background(), "owner", "aggregator") //nolint:staticcheck
a.ctx, a.exit = context.WithCancel(aggregatorCtx)
Expand Down Expand Up @@ -1300,7 +1300,7 @@ func TestTryBuildFinalProof(t *testing.T) {
ethTxManager := mocks.NewEthTxManager(t)
etherman := mocks.NewEtherman(t)
proverMock := mocks.NewProverMock(t)
a, err := New(cfg, stateMock, ethTxManager, etherman)
a, err := New(cfg, stateMock, ethTxManager, etherman, nil, nil)
require.NoError(err)
aggregatorCtx := context.WithValue(context.Background(), "owner", "aggregator") //nolint:staticcheck
a.ctx, a.exit = context.WithCancel(aggregatorCtx)
Expand Down Expand Up @@ -1430,7 +1430,7 @@ func TestIsSynced(t *testing.T) {
ethTxManager := mocks.NewEthTxManager(t)
etherman := mocks.NewEtherman(t)
proverMock := mocks.NewProverMock(t)
a, err := New(cfg, stateMock, ethTxManager, etherman)
a, err := New(cfg, stateMock, ethTxManager, etherman, nil, nil)
require.NoError(err)
aggregatorCtx := context.WithValue(context.Background(), "owner", "aggregator") //nolint:staticcheck
a.ctx, a.exit = context.WithCancel(aggregatorCtx)
Expand Down
23 changes: 23 additions & 0 deletions aggregator/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,17 @@ import (
"github.com/0xPolygonHermez/zkevm-node/encoding"
)

// SettlementBackend is the type of the settlement backend
type SettlementBackend string

const (
// AggLayer settlement backend
AggLayer SettlementBackend = "agglayer"

// L1 settlement backend
L1 SettlementBackend = "l1"
)

// TokenAmountWithDecimals is a wrapper type that parses token amount with decimals to big int
type TokenAmountWithDecimals struct {
*big.Int `validate:"required"`
Expand Down Expand Up @@ -88,4 +99,16 @@ type Config struct {

// UpgradeEtrogBatchNumber is the number of the first batch after upgrading to etrog
UpgradeEtrogBatchNumber uint64 `mapstructure:"UpgradeEtrogBatchNumber"`

// SettlementBackend configuration defines how a final ZKP should be settled. Directly to L1 or over the Beethoven service.
SettlementBackend SettlementBackend `mapstructure:"SettlementBackend"`

// AggLayerTxTimeout is the interval time to wait for a tx to be mined from the agglayer
AggLayerTxTimeout types.Duration `mapstructure:"AggLayerTxTimeout"`

// AggLayerURL url of the agglayer service
AggLayerURL string `mapstructure:"AggLayerURL"`

// SequencerPrivateKey Private key of the trusted sequencer
SequencerPrivateKey types.KeystoreFileConfig `mapstructure:"SequencerPrivateKey"`
}
1 change: 1 addition & 0 deletions aggregator/interfaces.go
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ type ethTxManager interface {

// etherman contains the methods required to interact with ethereum
type etherman interface {
GetRollupId() uint32
GetLatestVerifiedBatchNum() (uint64, error)
BuildTrustedVerifyBatchesTxData(lastVerifiedBatch, newVerifiedBatch uint64, inputs *ethmanTypes.FinalProofInputs, beneficiary common.Address) (to *common.Address, data []byte, err error)
}
Expand Down
20 changes: 19 additions & 1 deletion aggregator/mocks/mock_etherman.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading